{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T21:20:50Z","timestamp":1776201650026,"version":"3.50.1"},"reference-count":57,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2021,6,14]],"date-time":"2021-06-14T00:00:00Z","timestamp":1623628800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,14]],"date-time":"2021-06-14T00:00:00Z","timestamp":1623628800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"crossref","award":["EP\/N03368X\/1"],"award-info":[{"award-number":["EP\/N03368X\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2021,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Reinforcement learning was carried out in a simulated environment to learn continuous velocity control over multiple motor axes. This was then applied to a real-world optical tweezers experiment with the objective of moving a laser-trapped microsphere to a target location whilst avoiding collisions with other free-moving microspheres. The concept of training a neural network in a virtual environment has significant potential in the application of machine learning for experimental optimization and control, as the neural network can discover optimal methods for problem solving without the risk of damage to equipment, and at a speed not limited by movement in the physical environment. As the neural network treats both virtual and physical environments equivalently, we show that the network can also be applied to an augmented environment, where a virtual environment is combined with the physical environment. This technique may have the potential to unlock capabilities associated with mixed and augmented reality, such as enforcing safety limits for machine motion or as a method of inputting observations from additional sensors.<\/jats:p>","DOI":"10.1088\/2632-2153\/abf0f6","type":"journal-article","created":{"date-parts":[[2021,3,22]],"date-time":"2021-03-22T22:32:03Z","timestamp":1616452323000},"page":"035024","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Playing optical tweezers with deep reinforcement learning: in virtual, physical and augmented environments"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5814-6155","authenticated-orcid":false,"given":"Matthew","family":"Praeger","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8841-7235","authenticated-orcid":false,"given":"Yunhui","family":"Xie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4270-4247","authenticated-orcid":false,"given":"James A","family":"Grant-Jacob","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9704-2204","authenticated-orcid":false,"given":"Robert W","family":"Eason","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1784-1012","authenticated-orcid":false,"given":"Ben","family":"Mills","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2021,6,14]]},"reference":[{"key":"mlstabf0f6bib1","doi-asserted-by":"publisher","first-page":"1437","DOI":"10.1364\/OPTICA.4.001437","article-title":"Deep learning microscopy","volume":"4","author":"Rivenson","year":"2017","journal-title":"Optica"},{"key":"mlstabf0f6bib2","doi-asserted-by":"publisher","DOI":"10.1088\/2399-6528\/ab267d","article-title":"A neural lens for super-resolution biological imaging","volume":"3","author":"Grant-Jacob","year":"2019","journal-title":"J. Phys. Commun."},{"key":"mlstabf0f6bib3","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-53405-w","article-title":"Deep-UV excitation fluorescence microscopy for detection of lymph node metastasis using deep neural network","volume":"9","author":"Matsumoto","year":"2019","journal-title":"Sci. Rep."},{"key":"mlstabf0f6bib4","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1038\/s41377-018-0074-1","article-title":"Multimode optical fiber transmission with a deep learning network","volume":"7","author":"Rahmani","year":"2018","journal-title":"Light: Sci. Appl."},{"key":"mlstabf0f6bib5","doi-asserted-by":"publisher","first-page":"4923","DOI":"10.1038\/s41467-018-07355-y","article-title":"Machine learning analysis of extreme events in optical fibre modulation instability","volume":"9","author":"N\u00e4rhi","year":"2018","journal-title":"Nat. Commun."},{"key":"mlstabf0f6bib6","doi-asserted-by":"publisher","first-page":"7430","DOI":"10.1038\/s41598-017-07754-z","article-title":"Neuromorphic photonic networks using silicon photonic weight banks","volume":"7","author":"Tait","year":"2017","journal-title":"Sci. Rep."},{"key":"mlstabf0f6bib7","doi-asserted-by":"publisher","first-page":"1368","DOI":"10.1038\/s41598-018-37952-2","article-title":"Deep neural network inverse design of integrated photonic power splitters","volume":"9","author":"Tahersima","year":"2019","journal-title":"Sci. Rep."},{"key":"mlstabf0f6bib8","doi-asserted-by":"publisher","first-page":"27237","DOI":"10.1364\/OE.26.027237","article-title":"Real-time particle pollution sensing using machine learning","volume":"26","author":"Grant-Jacob","year":"2018","journal-title":"Opt. Express"},{"key":"mlstabf0f6bib9","doi-asserted-by":"publisher","first-page":"1235","DOI":"10.1364\/OL.43.001235","article-title":"Machine learning for improved image-based wavefront sensing","volume":"43","author":"Paine","year":"2018","journal-title":"Opt. Lett."},{"key":"mlstabf0f6bib10","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1038\/s41586-020-2038-x","article-title":"Ultrafast machine vision with 2D material neural network image sensors","volume":"579","author":"Mennel","year":"2020","journal-title":"Nature"},{"key":"mlstabf0f6bib11","doi-asserted-by":"publisher","first-page":"21574","DOI":"10.1364\/OE.26.021574","article-title":"Machine learning for 3D simulated visualization of laser machining","volume":"26","author":"Heath","year":"2018","journal-title":"Opt. Express"},{"key":"mlstabf0f6bib12","doi-asserted-by":"publisher","first-page":"17245","DOI":"10.1364\/OE.26.017245","article-title":"Predictive capabilities for laser machining via a neural network","volume":"26","author":"Mills","year":"2018","journal-title":"Opt. Express"},{"key":"mlstabf0f6bib13","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1038\/s41377-019-0192-4","article-title":"Emerging role of machine learning in light\u2013matter interaction","volume":"8","author":"Zhou","year":"2019","journal-title":"Light: Sci. Appl."},{"key":"mlstabf0f6bib14","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/abae76","article-title":"Machine learning reveals complex behaviours in optically trapped particles","volume":"1","author":"Lenton","year":"2020","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstabf0f6bib15","article-title":"Deep reinforcement learning that matters","author":"Henderson","year":"2017"},{"key":"mlstabf0f6bib16","article-title":"Deep reinforcement learning: an overview","author":"Li","year":"2017"},{"key":"mlstabf0f6bib17","author":"Sutton","year":"2018"},{"key":"mlstabf0f6bib18","article-title":"Reinforcement learning of artificial microswimmers","author":"Mui\u00f1os-Landin","year":"2018"},{"key":"mlstabf0f6bib19","article-title":"Learning from delayed rewards","author":"Watkins","year":"1989"},{"key":"mlstabf0f6bib20","article-title":"OpenAI gym","author":"Brockman","year":"2016"},{"key":"mlstabf0f6bib21","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2019.01291","article-title":"Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks","author":"James","year":"2019"},{"key":"mlstabf0f6bib22","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/0278364919887447","article-title":"Learning dexterous in-hand manipulation","volume":"39","author":"Andrychowicz","year":"2019","journal-title":"Int. J. Rob. Res."},{"key":"mlstabf0f6bib23","article-title":"Solving Rubik\u2019s cube with a robot hand","author":"Akkaya","year":"2019"},{"key":"mlstabf0f6bib24","article-title":"CAD2RL: real single-image flight without a single real image","author":"Sadeghi","year":"2016"},{"key":"mlstabf0f6bib25","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1109\/IROS.2018.8593706","article-title":"Laser-based reactive navigation for multirotor aerial robots using deep reinforcement learning","author":"Sampedro","year":"2018"},{"key":"mlstabf0f6bib26","article-title":"Virtual to real reinforcement learning for autonomous driving","author":"You","year":"2017"},{"key":"mlstabf0f6bib27","article-title":"Playing Atari with deep reinforcement learning","author":"Mnih","year":"2013"},{"key":"mlstabf0f6bib28","article-title":"Playing doom with slam-augmented deep reinforcement learning","author":"Bhatti","year":"2016"},{"key":"mlstabf0f6bib29","article-title":"Playing FPS games with deep reinforcement learning","author":"Lample","year":"2016"},{"key":"mlstabf0f6bib30","article-title":"StarCraft II: a new challenge for reinforcement learning","author":"Vinyals","year":"2017"},{"key":"mlstabf0f6bib31","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","article-title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","volume":"575","author":"Vinyals","year":"2019","journal-title":"Nature"},{"key":"mlstabf0f6bib32","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1126\/science.aau6249","article-title":"Human-level performance in 3D multiplayer games with population-based reinforcement learning","volume":"364","author":"Jaderberg","year":"2019","journal-title":"Science"},{"key":"mlstabf0f6bib33","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/D15-1001","article-title":"Language understanding for text-based games using deep reinforcement learning","author":"Narasimhan","year":"2015"},{"key":"mlstabf0f6bib34","article-title":"Playing text-adventure games with graph-based deep reinforcement learning","author":"Ammanabrolu","year":"2018"},{"key":"mlstabf0f6bib35","doi-asserted-by":"publisher","DOI":"10.1109\/CIG.2019.8848075","article-title":"Rogue-Gym: a new challenge for generalization in reinforcement learning","author":"Kanagawa","year":"2019"},{"key":"mlstabf0f6bib36","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"mlstabf0f6bib37","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","volume":"362","author":"Silver","year":"2018","journal-title":"Science"},{"key":"mlstabf0f6bib38","article-title":"Mastering atari, go, chess and shogi by planning with a learned model","author":"Schrittwieser","year":"2019"},{"key":"mlstabf0f6bib39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.mechatronics.2015.09.004","article-title":"Intelligent laser welding through representation, prediction, and control learning: an architecture with deep neural networks and reinforcement learning","volume":"34","author":"G\u00fcnther","year":"2016","journal-title":"Mechatronics"},{"key":"mlstabf0f6bib40","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/abb6d6","article-title":"Deep reinforcement learning for optical systems: a case study of mode-locked lasers","volume":"1","author":"Sun","year":"2020","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstabf0f6bib41","article-title":"Interferobot: aligning an optical interferometer by a reinforcement learning agent","author":"Sorokin","year":"2020"},{"key":"mlstabf0f6bib42","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1016\/j.ijleo.2018.09.160","article-title":"Self-learning control for wavefront sensorless adaptive optics system through deep reinforcement learning","volume":"178","author":"Ke","year":"2019","journal-title":"Optik"},{"key":"mlstabf0f6bib43","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1364\/JOCN.11.000547","article-title":"Routing in optical transport networks with deep reinforcement learning","volume":"11","author":"Suarez-Varela","year":"2019","journal-title":"IEEE\/OSA J. Opt. Commun. Netw."},{"key":"mlstabf0f6bib44","doi-asserted-by":"publisher","first-page":"24223","DOI":"10.1364\/OE.27.024223","article-title":"Deep reinforcement learning for coherent beam combining applications","volume":"27","author":"T\u00fcnnermann","year":"2019","journal-title":"Opt. Express"},{"key":"mlstabf0f6bib45","doi-asserted-by":"publisher","first-page":"810","DOI":"10.1038\/nature01935","article-title":"A revolution in optical manipulation","volume":"424","author":"Grier","year":"2003","journal-title":"Nature"},{"key":"mlstabf0f6bib46","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1016\/0734-189X(85)90016-7","article-title":"Topological structural analysis of digitized binary images by border following","volume":"30","author":"Suzuki","year":"1985","journal-title":"Comput. Vision Graph. Image Process."},{"key":"mlstabf0f6bib47","article-title":"Addressing function approximation error in actor-critic methods","author":"Fujimoto","year":"2018"},{"key":"mlstabf0f6bib48","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"mlstabf0f6bib49","article-title":"Deep residual learning for image recognition","author":"He","year":"2015"},{"key":"mlstabf0f6bib50","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2014"},{"key":"mlstabf0f6bib51","article-title":"Activation functions: comparison of trends in practice and research for deep learning","author":"Nwankpa","year":"2018"},{"key":"mlstabf0f6bib52","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1007\/BF00992699","article-title":"Self-improving reactive agents based on reinforcement learning, planning and teaching","volume":"8","author":"Lin","year":"1992","journal-title":"Mach. Learn."},{"key":"mlstabf0f6bib53","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap","year":"2015"},{"key":"mlstabf0f6bib54","article-title":"Concrete problems in AI safety","author":"Amodei","year":"2016"},{"key":"mlstabf0f6bib55","doi-asserted-by":"publisher","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick","year":"2017","journal-title":"Proc. Natl Acad. Sci."},{"key":"mlstabf0f6bib56","doi-asserted-by":"publisher","DOI":"10.1016\/j.physd.2019.132306","article-title":"Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network","volume":"404","author":"Sherstinsky","year":"2020","journal-title":"Physica D"},{"key":"mlstabf0f6bib57","doi-asserted-by":"publisher","first-page":"59","DOI":"10.3389\/frobt.2019.00059","article-title":"Automatic off-line design of robot swarms: a manifesto","volume":"6","author":"Birattari","year":"2019","journal-title":"Front. Robot. AI"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,13]],"date-time":"2021-12-13T15:44:09Z","timestamp":1639410249000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abf0f6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,14]]},"references-count":57,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,6,14]]},"published-print":{"date-parts":[[2021,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/abf0f6","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,14]]},"assertion":[{"value":"Playing optical tweezers with deep reinforcement learning: in virtual, physical and augmented environments","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2021 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2021-01-07","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-03-22","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-06-14","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}