{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T06:13:14Z","timestamp":1782799994880,"version":"3.54.5"},"reference-count":43,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2021,2,3]],"date-time":"2021-02-03T00:00:00Z","timestamp":1612310400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,3]],"date-time":"2021-02-03T00:00:00Z","timestamp":1612310400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100006801","name":"Horace H. Rackham School of Graduate Studies, University of Michigan","doi-asserted-by":"crossref","award":["Graduate Research Grant"],"award-info":[{"award-number":["Graduate Research Grant"]}],"id":[{"id":"10.13039\/100006801","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2021,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Optical multi-layer thin films are widely used in optical and energy applications requiring photonic designs. Engineers often design such structures based on their physical intuition. However, solely relying on human experts can be time-consuming and may lead to sub-optimal designs, especially when the design space is large. In this work, we frame the multi-layer optical design task as a sequence generation problem. A deep sequence generation network is proposed for efficiently generating optical layer sequences. We train the deep sequence generation network with proximal policy optimization to generate multi-layer structures with desired properties. The proposed method is applied to two energy applications. Our algorithm successfully discovered high-performance designs, outperforming structures designed by human experts in task 1, and a state-of-the-art memetic algorithm in task 2.<\/jats:p>","DOI":"10.1088\/2632-2153\/abc327","type":"journal-article","created":{"date-parts":[[2020,10,20]],"date-time":"2020-10-20T22:45:38Z","timestamp":1603233938000},"page":"025013","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":60,"title":["Automated multi-layer optical design via deep reinforcement learning"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9679-0144","authenticated-orcid":false,"given":"Haozhu","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zeyu","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengang","family":"Ji","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"L","family":"Jay Guo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2021,2,3]]},"reference":[{"key":"mlstabc327bib1","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1021\/acsphotonics.5b00689","article-title":"Compact multilayer film structures for ultrabroadband, omnidirectional and efficient absorption","volume":"3","author":"Yang","year":"2016","journal-title":"ACS Photonics"},{"key":"mlstabc327bib2","doi-asserted-by":"publisher","first-page":"5385","DOI":"10.1364\/OE.16.005385","article-title":"Broadband optical absorption enhancement through coherent light trapping in thin-film photovoltaic cells","volume":"16","author":"Agrawal","year":"2008","journal-title":"Opt. Express"},{"key":"mlstabc327bib3","doi-asserted-by":"publisher","first-page":"540","DOI":"10.1038\/nature13883","article-title":"Passive radiative cooling below ambient air temperature under direct sunlight","volume":"515","author":"Raman","year":"2014","journal-title":"Nature"},{"key":"mlstabc327bib4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-018-06535-0","article-title":"Photonic thermal management of coloured objects","volume":"9","author":"Wei","year":"2018","journal-title":"Nat. Commun."},{"key":"mlstabc327bib5","doi-asserted-by":"publisher","first-page":"5290","DOI":"10.1364\/OE.16.005290","article-title":"Design of multilayer antireflection coatings made from co-sputtered and low-refractive-index materials by genetic algorithm","volume":"16","author":"Schubert","year":"2008","journal-title":"Opt. Express"},{"key":"mlstabc327bib6","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1021\/acsphotonics.7b01136","article-title":"Optimization of multilayer optical films with a memetic algorithm and mixed integer programming","volume":"5","author":"Shi","year":"2017","journal-title":"ACS Photonics"},{"key":"mlstabc327bib7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/JPHOT.2020.2994647","article-title":"Optimized multilayer structures with ultrabroadband near-perfect absorption","volume":"12","author":"You","year":"2020","journal-title":"IEEE Photonics J."},{"key":"mlstabc327bib8","doi-asserted-by":"publisher","first-page":"5493","DOI":"10.1364\/AO.35.005493","article-title":"Application of the needle optimization technique to the design of optical coatings","volume":"35","author":"Tikhonravov","year":"1996","journal-title":"Appl. Opt."},{"key":"mlstabc327bib9","doi-asserted-by":"publisher","first-page":"548","DOI":"10.1016\/j.ijleo.2013.07.028","article-title":"Global optimal design of optical multilayer thin-film filters using particle swarm optimization","volume":"125","author":"Rabady","year":"2014","journal-title":"Optik"},{"key":"mlstabc327bib10","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"mlstabc327bib11","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","article-title":"Grandmaster level in Starcraft II using multi-agent reinforcement learning","volume":"575","author":"Vinyals","year":"2019","journal-title":"Nature"},{"key":"mlstabc327bib12","article-title":"Neural combinatorial optimization with reinforcement learning","author":"Bello","year":"2016"},{"key":"mlstabc327bib13","first-page":"6348","article-title":"Learning combinatorial optimization algorithms over graphs","author":"Khalil","year":"2017"},{"key":"mlstabc327bib14","first-page":"2430","article-title":"Device placement optimization with reinforcement learning","volume":"vol 70","author":"Mirhoseini","year":"2017"},{"key":"mlstabc327bib15","article-title":"Chip placement with deep reinforcement learning","author":"Mirhoseini","year":"2020"},{"key":"mlstabc327bib16","author":"Sutton","year":"2018"},{"key":"mlstabc327bib17","article-title":"A learning-based iterative method for solving vehicle routing problems","author":"Hao","year":"2020"},{"key":"mlstabc327bib18","doi-asserted-by":"publisher","first-page":"5874","DOI":"10.1364\/OE.27.005874","article-title":"Optimisation of colour generation from dielectric nanostructures using reinforcement learning","volume":"27","author":"Sajedian","year":"2019","journal-title":"Opt. Express"},{"key":"mlstabc327bib19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-019-47154-z","article-title":"Double-deep q-learning to increase the efficiency of metasurface holograms","volume":"9","author":"Sajedian","year":"2019","journal-title":"Sci. Rep."},{"key":"mlstabc327bib20","doi-asserted-by":"publisher","first-page":"6326","DOI":"10.1021\/acsnano.8b03569","article-title":"Deep-learning-enabled on-demand design of chiral metamaterials","volume":"12","author":"Wei","year":"2018","journal-title":"ACS Nano"},{"key":"mlstabc327bib21","doi-asserted-by":"publisher","first-page":"1365","DOI":"10.1021\/acsphotonics.7b01377","article-title":"Training deep neural networks for the inverse design of nanophotonic structures","volume":"5","author":"Liu","year":"2018","journal-title":"ACS Photonics"},{"key":"mlstabc327bib22","doi-asserted-by":"publisher","first-page":"6570","DOI":"10.1021\/acs.nanolett.8b03171","article-title":"Generative model for the inverse design of metasurfaces","volume":"18","author":"Liu","year":"2018","journal-title":"Nano Lett."},{"key":"mlstabc327bib23","first-page":"2692","article-title":"Pointer networks","author":"Vinyals","year":"2015"},{"key":"mlstabc327bib24","first-page":"6278","article-title":"Learning to perform local rewriting for combinatorial optimization","author":"Chen","year":"2019"},{"key":"mlstabc327bib25","first-page":"1192","article-title":"Deep reinforcement learning for dialogue generation","author":"Jiwei","year":"2016"},{"key":"mlstabc327bib26","doi-asserted-by":"publisher","first-page":"eaa7885","DOI":"10.1126\/sciadv.aap7885","article-title":"Deep reinforcement learning for de novo drug design","volume":"4","author":"Popova","year":"2018","journal-title":"Sci. Adv."},{"key":"mlstabc327bib27","article-title":"Model-based reinforcement learning for biological sequence design","author":"Angermueller","year":"2020"},{"key":"mlstabc327bib28","doi-asserted-by":"publisher","first-page":"8872","DOI":"10.1021\/acsnano.9b02371","article-title":"Free-form diffractive metagrating design based on generative adversarial networks","volume":"13","author":"Jiang","year":"2019","journal-title":"ACS Nano"},{"key":"mlstabc327bib29","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"mlstabc327bib30","article-title":"Generating sequences with recurrent neural networks","author":"Graves","year":"2013"},{"key":"mlstabc327bib31","article-title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","author":"Chung","year":"2014"},{"key":"mlstabc327bib32","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1145\/279232.279236","article-title":"Algorithm 778: L-bfgs-b: fortran subroutines for large-scale bound-constrained optimization","volume":"23","author":"Zhu","year":"1997","journal-title":"ACM Trans. Math. Softw. (TOMS)"},{"key":"mlstabc327bib33","author":"Goodfellow","year":"2016"},{"key":"mlstabc327bib34","article-title":"Multilayer optical calculations","author":"Byrnes","year":"2016"},{"key":"mlstabc327bib35","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017"},{"key":"mlstabc327bib36","article-title":"High-dimensional continuous control using generalized advantage estimation","author":"Schulman","year":"2015"},{"key":"mlstabc327bib37","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"mlstabc327bib38","first-page":"8024","article-title":"Pytorch: an imperative style, high-performance deep learning library","author":"Paszke","year":"2019"},{"key":"mlstabc327bib39","author":"Achiam","year":"2018"},{"key":"mlstabc327bib40","doi-asserted-by":"publisher","first-page":"3017","DOI":"10.1002\/adma.201505451","article-title":"Efficient thermal\u2013light interconversions based on optical topological transition in the metal-dielectric multilayered metamaterials","volume":"28","author":"Zhou","year":"2016","journal-title":"Adv. Mater."},{"key":"mlstabc327bib41","doi-asserted-by":"publisher","first-page":"320","DOI":"10.1038\/nnano.2015.309","article-title":"Tailoring high-temperature radiation and the resurrection of the incandescent source","volume":"11","author":"Ilic","year":"2016","journal-title":"Nat. Nanotechnol."},{"key":"mlstabc327bib42","doi-asserted-by":"publisher","first-page":"774","DOI":"10.1021\/acsphotonics.7b00089","article-title":"A comprehensive photonic approach for solar cell cooling","volume":"4","author":"Wei","year":"2017","journal-title":"ACS Photonics"},{"key":"mlstabc327bib43","doi-asserted-by":"publisher","first-page":"15995","DOI":"10.1364\/OE.26.015995","article-title":"Nanophotonic control of thermal radiation for energy applications","volume":"26","author":"Wei","year":"2018","journal-title":"Opt. Express"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,1,15]],"date-time":"2022-01-15T05:41:30Z","timestamp":1642225290000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc327"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,3]]},"references-count":43,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,2,3]]},"published-print":{"date-parts":[[2021,6,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/abc327","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,3]]},"assertion":[{"value":"Automated multi-layer optical design via deep reinforcement learning","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2021 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2020-06-30","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-10-20","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-02-03","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}