{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T20:21:58Z","timestamp":1767990118897,"version":"3.49.0"},"reference-count":51,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T00:00:00Z","timestamp":1744761600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T00:00:00Z","timestamp":1744761600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100011665","name":"Thomas Jefferson National Accelerator Facility","doi-asserted-by":"crossref","award":["DE-AC05-06OR23177"],"award-info":[{"award-number":["DE-AC05-06OR23177"]}],"id":[{"id":"10.13039\/100011665","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100011664","name":"SLAC National Accelerator Laboratory","doi-asserted-by":"crossref","award":["DE-AC02-76SF00515"],"award-info":[{"award-number":["DE-AC02-76SF00515"]}],"id":[{"id":"10.13039\/100011664","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Particle accelerator operation requires simultaneous optimization of multiple objectives. Multi-objective optimization (MOO) is particularly challenging due to trade-offs between the objectives. Evolutionary algorithms, such as genetic algorithms (GAs), have been leveraged for many optimization problems, however, they do not apply to complex control problems by design. This paper demonstrates the power of differentiability for solving MOO problems in particle accelerators using a deep differentiable reinforcement learning (DDRL) algorithm. We compare the DDRL algorithm with model-free reinforcement learning (MFRL), GA, and Bayesian optimization (BO) for simultaneous optimization of heat load and trip rates in the continuous electron beam accelerator facility. The underlying problem enforces strict constraints on both individual states and actions as well as cumulative (global) constraints on energy requirements of the beam. Using historical accelerator data, we develop a physics-based surrogate model which is differentiable and allows for back-propagation of gradients. The results are evaluated in the form of a Pareto-front with two objectives. We show that the DDRL outperforms MFRL, BO, and GA on high dimensional problems.<\/jats:p>","DOI":"10.1088\/2632-2153\/adc221","type":"journal-article","created":{"date-parts":[[2025,3,18]],"date-time":"2025-03-18T22:56:16Z","timestamp":1742338576000},"page":"025018","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Harnessing the power of gradient-based simulations for multi-objective optimization in particle accelerators"],"prefix":"10.1088","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4430-9937","authenticated-orcid":true,"given":"Kishansingh","family":"Rajput","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3475-2871","authenticated-orcid":true,"given":"Malachi","family":"Schram","sequence":"additional","affiliation":[]},{"given":"Auralee","family":"Edelen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4162-0276","authenticated-orcid":true,"given":"Jonathan","family":"Colen","sequence":"additional","affiliation":[]},{"given":"Armen","family":"Kasparian","sequence":"additional","affiliation":[]},{"given":"Ryan","family":"Roussel","sequence":"additional","affiliation":[]},{"given":"Adam","family":"Carpenter","sequence":"additional","affiliation":[]},{"given":"He","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Jay","family":"Benesch","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2025,4,16]]},"reference":[{"key":"mlstadc221bib1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.27.084802","volume":"27","author":"Adderley","year":"2024","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib2","article-title":"Addressing function approximation error in actor-critic methods","author":"Fujimoto","year":"2018"},{"key":"mlstadc221bib3","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1109\/4235.996017","volume":"6","author":"Deb","year":"2002","journal-title":"IEEE Trans. Evol. Comput."},{"key":"mlstadc221bib4","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.24.062801","volume":"24","author":"Roussel","year":"2021","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib5","doi-asserted-by":"crossref","DOI":"10.1080\/14697688.2022.2062431","article-title":"Deep differentiable reinforcement learning and optimal trading","author":"Jaisson","year":"2022"},{"key":"mlstadc221bib6","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ad2e18","volume":"5","author":"Rajput","year":"2024","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstadc221bib7","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.25.122802","volume":"25","author":"Blokland","year":"2022","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib8","article-title":"Anomaly detection in particle accelerators using autoencoders","author":"Edelen","year":"2021"},{"key":"mlstadc221bib9","doi-asserted-by":"publisher","DOI":"10.1016\/j.mlwa.2023.100484","volume":"13","author":"Alanazi","year":"2023","journal-title":"Mach. Learn. Appl."},{"key":"mlstadc221bib10","first-page":"pp 240","article-title":"Machine learning-based anomaly detection for particle accelerators","author":"Marcato","year":"2021"},{"key":"mlstadc221bib11","doi-asserted-by":"crossref","DOI":"10.2172\/1825276","article-title":"Developing robust digital twins and reinforcement learning for accelerator control systems at the fermilab booster","author":"Kafkes","year":"2021"},{"key":"mlstadc221bib12","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1103\/PhysRevAccelBeams.26.044602","volume":"26","author":"Schram","year":"2023","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib13","volume":"16","author":"Scheinker","year":"2013","journal-title":"Phys. Rev. Spec. Top."},{"key":"mlstadc221bib14","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1063\/5.0003423","volume":"10","author":"Scheinker","year":"2020","journal-title":"AIP Adv."},{"key":"mlstadc221bib15","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevSTAB.17.101003","volume":"17","author":"c v","year":"2014","journal-title":"Phys. Rev. Spec. Top."},{"key":"mlstadc221bib16","doi-asserted-by":"publisher","DOI":"10.1016\/j.nima.2024.169702","volume":"1067","author":"Huber","year":"2024","journal-title":"Nucl. Instrum. Methods Phys. Res. A"},{"key":"mlstadc221bib17","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.22.054602","volume":"22","author":"Neveu","year":"2019","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib18","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.23.044601","volume":"23","author":"Edelen","year":"2020","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib19","doi-asserted-by":"crossref","DOI":"10.1103\/PhysRevAccelBeams.24.014601","volume":"24","author":"Kranj\u010devi\u0107","year":"2021","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib20","author":"Boltz","year":"2024"},{"key":"mlstadc221bib21","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.24.072802","volume":"24","author":"Hanuka","year":"2021","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib22","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.27.084801","volume":"27","author":"Roussel","year":"2024","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib23","doi-asserted-by":"publisher","first-page":"842","DOI":"10.18429\/JACoW-NAPAC2022-THXD4","volume":"22","author":"Kuklev","year":"2022","journal-title":"Proc. NAPAC"},{"key":"mlstadc221bib24","doi-asserted-by":"publisher","first-page":"4726","DOI":"10.1038\/s41467-024-48923-9","volume":"15","author":"Ji","year":"2024","journal-title":"Nat. Commun."},{"key":"mlstadc221bib25","first-page":"pp 707","article-title":"Bayesian optimisation for fast and safe parameter tuning of swissfel","author":"Kirschner","year":"2019"},{"key":"mlstadc221bib26","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.26.084601","volume":"26","author":"Ferran","year":"2023","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib27","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.24.104601","volume":"24","author":"St","year":"2021","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib28","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-66263-y","volume":"14","author":"Kaiser","year":"2024","journal-title":"Sci. Rep."},{"key":"mlstadc221bib29","first-page":"pp 1808","article-title":"Preliminary results on the reinforcement learning-based control of the microbunching instability","author":"Scomparin","year":"2024"},{"key":"mlstadc221bib30","first-page":"pp 1804","article-title":"Towards few-shot reinforcement learning in particle accelerator control","author":"Hirlaender","year":"2024"},{"key":"mlstadc221bib31","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2021.718950","volume":"4","author":"Vidyaratne","year":"2022","journal-title":"Front. Artif. Intell."},{"key":"mlstadc221bib32","article-title":"A longitudinal study of field emission in CEBAF\u2019s SRF cavities 1995-2015","author":"Benesch","year":"2015"},{"key":"mlstadc221bib33","article-title":"A toolkit for reliable benchmarking and research in multi-objective reinforcement learning","author":"Felten","year":"2023"},{"key":"mlstadc221bib34","article-title":"TensorFlow: large-scale machine learning on heterogeneous systems software","author":"Abadi","year":"2015"},{"key":"mlstadc221bib35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3453474","volume":"54","author":"Guerreiro","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"mlstadc221bib36","doi-asserted-by":"publisher","first-page":"89497","DOI":"10.1109\/ACCESS.2020.2990567","volume":"8","author":"Blank","year":"2020","journal-title":"IEEE Access"},{"key":"mlstadc221bib37","author":"Rasmussen","year":"2006"},{"key":"mlstadc221bib38","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.124.124801","volume":"124","author":"Duris","year":"2020","journal-title":"Phys. Rev. Lett."},{"key":"mlstadc221bib39","first-page":"pp 45","author":"Goan","year":"2020"},{"key":"mlstadc221bib40","author":"Eriksson","year":"2019"},{"key":"mlstadc221bib41","volume":"vol 107","author":"Pardalos","year":"2016"},{"key":"mlstadc221bib42","article-title":"Parallel bayesian optimization of multiple noisy objectives with expected hypervolume improvement","author":"Daulton","year":"2021"},{"key":"mlstadc221bib43","first-page":"pp 4796","article-title":"Xopt: a simplified framework for optimization of accelerator problems using advanced algorithms","author":"Roussel","year":"2023"},{"key":"mlstadc221bib44","article-title":"BoTorch: a framework for efficient Monte-Carlo Bayesian optimization","volume":"vol 33","author":"Balandat","year":"2020"},{"key":"mlstadc221bib45","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1162\/106365600568202","volume":"8","author":"Zitzler","year":"2000","journal-title":"Evol. Comput."},{"key":"mlstadc221bib46","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1007\/s10458-022-09552-y","volume":"36","author":"Hayes","year":"2022","journal-title":"Auto. Agents Multi-Agent Syst."},{"key":"mlstadc221bib47","article-title":"PD-MORL: preference-driven multi-objective reinforcement learning algorithm","author":"Basaklar","year":"2023"},{"key":"mlstadc221bib48","doi-asserted-by":"crossref","DOI":"10.2139\/ssrn.3591734","article-title":"Differential machine learning","author":"Huge","year":"2020"},{"key":"mlstadc221bib49","article-title":"Practical Bayesian optimization of machine learning algorithms","volume":"vol 25","author":"Snoek","year":"2012"},{"key":"mlstadc221bib50","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.27.054601","volume":"27","author":"Kaiser","year":"2024","journal-title":"Phys. Rev. Accel. Beams"},{"key":"mlstadc221bib51","first-page":"p WEA065","volume":"vol IPAC2023","author":"Gonzalez-Aguilera","year":"2023"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T10:24:01Z","timestamp":1744799041000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/adc221"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,16]]},"references-count":51,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,4,16]]},"published-print":{"date-parts":[[2025,6,30]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/adc221","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,16]]},"assertion":[{"value":"Harnessing the power of gradient-based simulations for multi-objective optimization in particle accelerators","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-11-09","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-03-18","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-04-16","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}