{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T14:11:39Z","timestamp":1773411099527,"version":"3.50.1"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2021,11,16]],"date-time":"2021-11-16T00:00:00Z","timestamp":1637020800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2021,11,16]],"date-time":"2021-11-16T00:00:00Z","timestamp":1637020800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nat Mach Intell"],"DOI":"10.1038\/s42256-021-00411-1","type":"journal-article","created":{"date-parts":[[2021,11,16]],"date-time":"2021-11-16T17:02:38Z","timestamp":1637082158000},"page":"985-994","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Deep reinforcement learning-designed radiofrequency waveform in MRI"],"prefix":"10.1038","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2549-0136","authenticated-orcid":false,"given":"Dongmyung","family":"Shin","sequence":"first","affiliation":[]},{"given":"Younghoon","family":"Kim","sequence":"additional","affiliation":[]},{"given":"Chungseok","family":"Oh","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6431-6720","authenticated-orcid":false,"given":"Hongjun","family":"An","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0014-3410","authenticated-orcid":false,"given":"Juhyung","family":"Park","sequence":"additional","affiliation":[]},{"given":"Jiye","family":"Kim","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9485-5434","authenticated-orcid":false,"given":"Jongho","family":"Lee","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,11,16]]},"reference":[{"key":"411_CR1","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","volume":"550","author":"D Silver","year":"2017","unstructured":"Silver, D. et al. Mastering the game of Go without human knowledge. Nature 550, 354\u2013359 (2017).","journal-title":"Nature"},{"key":"411_CR2","unstructured":"Wu, Y. et al. Google\u2019s neural machine translation system: bridging the gap between human and machine translation. Preprint at https:\/\/arxiv.org\/abs\/1609.08144 (2016)."},{"key":"411_CR3","doi-asserted-by":"crossref","unstructured":"Yu, J. et al. Generative image inpainting with contextual attention. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 5505\u20135514 (IEEE, 2018).","DOI":"10.1109\/CVPR.2018.00577"},{"key":"411_CR4","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1038\/nature25978","volume":"555","author":"MH Segler","year":"2018","unstructured":"Segler, M. H., Preuss, M. & Waller, M. P. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 604\u2013610 (2018).","journal-title":"Nature"},{"key":"411_CR5","unstructured":"Runge, F., Stoll, D., Falkner, S. & Hutter, F. Learning to design RNA. In Proc. International Conference on Learning Representations (2019)."},{"key":"411_CR6","unstructured":"Wang, H., Yang, J., Lee, H.-S. & Han, S. Learning to design circuits. In Proc. Conference on Neural Information Processing Systems (2018)."},{"key":"411_CR7","doi-asserted-by":"crossref","unstructured":"Popova, M., Isayev, O. & Tropsha, A. Deep reinforcement learning for de novo drug design. Sci. Adv. 4, aap7885 (2018).","DOI":"10.1126\/sciadv.aap7885"},{"key":"411_CR8","unstructured":"Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018)."},{"key":"411_CR9","doi-asserted-by":"publisher","first-page":"1578","DOI":"10.1109\/18.259642","volume":"39","author":"MR Bell","year":"1993","unstructured":"Bell, M. R. Information theory and radar waveform design. IEEE Trans. Inf. Theory 39, 1578\u20131597 (1993).","journal-title":"IEEE Trans. Inf. Theory"},{"key":"411_CR10","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1109\/58.753026","volume":"46","author":"DH Simpson","year":"1999","unstructured":"Simpson, D. H., Chin, C. T. & Burns, P. N. Pulse inversion Doppler: a new method for detecting nonlinear echoes from microbubble contrast agents. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 46, 372\u2013382 (1999).","journal-title":"IEEE Trans. Ultrason. Ferroelectr. Freq. Control"},{"key":"411_CR11","doi-asserted-by":"crossref","unstructured":"Tarantola, A. Inverse Problem Theory and Methods for Model Parameter Estimation (SIAM, 2005).","DOI":"10.1137\/1.9780898717921"},{"key":"411_CR12","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1103\/PhysRev.70.460","volume":"70","author":"F Bloch","year":"1946","unstructured":"Bloch, F. Nuclear induction. Phys. Rev. 70, 460 (1946).","journal-title":"Phys. Rev."},{"key":"411_CR13","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1109\/42.75611","volume":"10","author":"J Pauly","year":"1991","unstructured":"Pauly, J., Le Roux, P., Nishimura, D. & Macovski, A. Parameter relations for the Shinnar\u2013Le Roux selective excitation pulse design algorithm. IEEE Trans. Med. Imaging 10, 53\u201365 (1991).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"411_CR14","doi-asserted-by":"crossref","unstructured":"Abragam, A. The Principles of Nuclear Magnetism (Oxford Univ. Press, 1961).","DOI":"10.1063\/1.3057238"},{"key":"411_CR15","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1109\/TMI.1986.4307754","volume":"5","author":"S Conolly","year":"1986","unstructured":"Conolly, S., Nishimura, D. & Macovski, A. Optimal control solutions to the magnetic resonance selective excitation problem. IEEE Trans. Med. Imaging 5, 106\u2013115 (1986).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"411_CR16","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1016\/j.jmr.2004.11.004","volume":"172","author":"N Khaneja","year":"2005","unstructured":"Khaneja, N., Reiss, T., Kehlet, C., Schulte-Herbr\u00fcggen, T. & Glaser, S. J. Optimal control of coupled spin dynamics: design of NMR pulse sequences by gradient ascent algorithms. J. Magn. Reson. 172, 296\u2013305 (2005).","journal-title":"J. Magn. Reson."},{"key":"411_CR17","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1002\/mrm.1910360311","volume":"36","author":"D Rosenfeld","year":"1996","unstructured":"Rosenfeld, D. & Zur, Y. Design of adiabatic selective pulses using optimal control theory. Magn. Reson. Med. 36, 401\u2013409 (1996).","journal-title":"Magn. Reson. Med."},{"key":"411_CR18","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1109\/TMI.2017.2758391","volume":"37","author":"A Rund","year":"2017","unstructured":"Rund, A., Aigner, C. S., Kunisch, K. & Stollberger, R. Magnetic resonance RF pulse design by optimal control with physical constraints. IEEE Trans. Med. Imaging 37, 461\u2013472 (2017).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"411_CR19","doi-asserted-by":"publisher","first-page":"1416","DOI":"10.1002\/mrm.27124","volume":"80","author":"A Rund","year":"2018","unstructured":"Rund, A., Aigner, C. S., Kunisch, K. & Stollberger, R. Simultaneous multislice refocusing via time optimal control. Magn. Reson. Med. 80, 1416\u20131428 (2018).","journal-title":"Magn. Reson. Med."},{"key":"411_CR20","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1002\/mrm.21485","volume":"59","author":"D Xu","year":"2008","unstructured":"Xu, D., King, K. F., Zhu, Y., McKinnon, G. C. & Liang, Z. P. Designing multichannel, multidimensional, arbitrary flip angle RF pulses using an optimal control approach. Magn. Reson. Med. 59, 547\u2013560 (2008).","journal-title":"Magn. Reson. Med."},{"key":"411_CR21","doi-asserted-by":"publisher","first-page":"054203","DOI":"10.1063\/1.4739755","volume":"137","author":"MS Vinding","year":"2012","unstructured":"Vinding, M. S., Maximov, I. I., To\u0161ner, Z. & Nielsen, N. C. Fast numerical design of spatial-selective RF pulses in MRI using Krotov and quasi-Newton based optimal control methods. J. Chem. Phys. 137, 054203 (2012).","journal-title":"J. Chem. Phys."},{"key":"411_CR22","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1002\/mrm.26086","volume":"77","author":"MS Vinding","year":"2017","unstructured":"Vinding, M. S., Gu\u00e9rin, B., Vosegaard, T. & Nielsen, N. C. Local SAR, global SAR, and power\u2010constrained large\u2010flip\u2010angle pulses with optimal control and virtual observation points. Magn. Reson. Med. 77, 374\u2013384 (2017).","journal-title":"Magn. Reson. Med."},{"key":"411_CR23","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1002\/mrm.27716","volume":"82","author":"M Loecher","year":"2019","unstructured":"Loecher, M., Magrath, P., Aliotta, E. & Ennis, D. B. Time\u2010optimized 4D phase contrast MRI with real\u2010time convex optimization of gradient waveforms and fast excitation methods. Magn. Reson. Med. 82, 213\u2013224 (2019).","journal-title":"Magn. Reson. Med."},{"key":"411_CR24","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/j.jmr.2015.11.010","volume":"262","author":"H Shang","year":"2016","unstructured":"Shang, H. et al. Multiband RF pulses with improved performance via convex optimization. J. Magn. Reson. 262, 81\u201390 (2016).","journal-title":"J. Magn. Reson."},{"key":"411_CR25","doi-asserted-by":"crossref","unstructured":"Vinding, M. S., Aigner, C. S., Schmitter, S. & Lund, T. E. DeepControl: 2D RF pulses facilitating B1+ inhomogeneity and B0 off-resonance compensation in vivo at 7T. Magn. Reson. Med. 85, 3308\u20133317 (2021).","DOI":"10.1002\/mrm.28667"},{"key":"411_CR26","doi-asserted-by":"publisher","first-page":"586","DOI":"10.1002\/mrm.27740","volume":"82","author":"MS Vinding","year":"2019","unstructured":"Vinding, M. S., Skyum, B., Sangill, R. & Lund, T. E. Ultrafast (milliseconds), multidimensional RF pulse design with deep learning. Magn. Reson. Med. 82, 586\u2013599 (2019).","journal-title":"Magn. Reson. Med."},{"key":"411_CR27","unstructured":"Mirfin, C., Glover, P. & Bowtell, R. Optimisation of parallel transmission radiofrequency pulses using neural networks. In Proc. 26th Annual Meeting of ISMRM (2018)."},{"key":"411_CR28","doi-asserted-by":"crossref","unstructured":"Zhang, Y. et al. Multi\u2010task convolutional neural network\u2010based design of radio frequency pulse and the accompanying gradients for magnetic resonance imaging. NMR Biomed. 34, e4443 (2021).","DOI":"10.1002\/nbm.4443"},{"key":"411_CR29","unstructured":"Goodfellow, I., Bengio, Y., Courville, A. & Bengio, Y. Deep Learning (MIT Press, 2016)."},{"key":"411_CR30","first-page":"347","volume":"59","author":"MS Silver","year":"1984","unstructured":"Silver, M. S., Joseph, R. & Hoult, D. Highly selective \u03c02 and \u03c0 pulse generation. J. Magn. Reson. 59, 347\u2013351 (1984).","journal-title":"J. Magn. Reson."},{"key":"411_CR31","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1006\/jmre.2001.2340","volume":"153","author":"M Garwood","year":"2001","unstructured":"Garwood, M. & DelaBarre, L. The return of the frequency sweep: designing adiabatic pulses for contemporary NMR. J. Magn. Reson. 153, 155\u2013177 (2001).","journal-title":"J. Magn. Reson."},{"key":"411_CR32","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1002\/(SICI)1099-1492(199712)10:8<423::AID-NBM488>3.0.CO;2-X","volume":"10","author":"A Tann\u00fas","year":"1997","unstructured":"Tann\u00fas, A. & Garwood, M. Adiabatic pulses. NMR Biomed. 10, 423\u2013434 (1997).","journal-title":"NMR Biomed."},{"key":"411_CR33","doi-asserted-by":"crossref","unstructured":"Henderson, P. et al. Deep reinforcement learning that matters. In Proc. AAAI Conference on Artificial Intelligence (2018).","DOI":"10.1609\/aaai.v32i1.11694"},{"key":"411_CR34","unstructured":"Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. International Conference on Artificial Intelligence and Statistics 249\u2013256 (2010)."},{"key":"411_CR35","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","volume":"575","author":"O Vinyals","year":"2019","unstructured":"Vinyals, O. et al. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575, 350\u2013354 (2019).","journal-title":"Nature"},{"key":"411_CR36","unstructured":"Zhu, B., Liu, J. Z., Koonjoo, N., Rose, B. R. & Rosen, M. S. Automated pulse sequence generation (AUTOSEQ) using Bayesian reinforcement learning in an MRI physics simulation environment. In Proc. 26th Annual Meeting of ISMRM (2018)."},{"key":"411_CR37","unstructured":"Walker-Samuel, S. Using deep reinforcement learning to actively, adaptively and autonomously control of a simulated MRI scanner. In Proc. 27th Annual Meeting of ISMRM (2019)."},{"key":"411_CR38","unstructured":"David, Y. et al. Reinforcement learning for online undersampling pattern optimization. In Proc. 27th Annual Meeting of ISMRM (2019)."},{"key":"411_CR39","doi-asserted-by":"crossref","unstructured":"Pineda, L., Basu, S., Romero, A., Calandra, R. & Drozdzal, M. Active MR k-space sampling with reinforcement learning. Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention (2020).","DOI":"10.1007\/978-3-030-59713-9_3"},{"key":"411_CR40","doi-asserted-by":"publisher","first-page":"1139","DOI":"10.1109\/TCI.2020.3006727","volume":"6","author":"CD Bahadir","year":"2020","unstructured":"Bahadir, C. D., Wang, A. Q., Dalca, A. V. & Sabuncu, M. R. Deep-learning-based optimization of the under-sampling pattern in MRI. IEEE Trans. Comput. Imaging 6, 1139\u20131152 (2020).","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"411_CR41","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1002\/mrm.1910150211","volume":"15","author":"CH Meyer","year":"1990","unstructured":"Meyer, C. H., Pauly, J. M., Macovskiand, A. & Nishimura, D. G. Simultaneous spatial and spectral selective excitation. Magn. Reson. Med. 15, 287\u2013304 (1990).","journal-title":"Magn. Reson. Med."},{"key":"411_CR42","doi-asserted-by":"publisher","first-page":"908","DOI":"10.1002\/mrm.20631","volume":"54","author":"CY Yip","year":"2005","unstructured":"Yip, C. Y., Fessler, J. A. & Noll, D. C. Iterative RF pulse design for multidimensional, small\u2010tip\u2010angle selective excitation. Magn. Reson. Med. 54, 908\u2013917 (2005).","journal-title":"Magn. Reson. Med."},{"key":"411_CR43","doi-asserted-by":"crossref","unstructured":"Cho, K. et al. Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation. In Proc. Conference on Empirical Methods in Natural Language Processing (2014).","DOI":"10.3115\/v1\/D14-1179"},{"key":"411_CR44","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O. Proximal policy optimization algorithms. Preprint at https:\/\/arxiv.org\/abs\/1707.06347 (2017)."},{"key":"411_CR45","unstructured":"Paszke, A. et al. Automatic differentiation in pytorch. In Proc. Conference on Neural Information Processing Systems (2017)."},{"key":"411_CR46","unstructured":"Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. International Conference for Learning Representations (2015)."},{"key":"411_CR47","doi-asserted-by":"publisher","first-page":"1205","DOI":"10.1016\/0730-725X(94)90086-7","volume":"12","author":"GB Matson","year":"1994","unstructured":"Matson, G. B. An integrated program for amplitude-modulated RF pulse generation and re-mapping with shaped gradients. Magn. Reson. Imaging 12, 1205\u20131225 (1994).","journal-title":"Magn. Reson. Imaging"},{"key":"411_CR48","unstructured":"Martin, J. B. et al. SigPy.RF: comprehensive open-source RF pulse design tools for reproducible research. In Proc. 28th Annual Meeting of ISMRM (2020)."},{"key":"411_CR49","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1002\/mrm.25587","volume":"75","author":"JP Stockmann","year":"2016","unstructured":"Stockmann, J. P. et al. A 32\u2010channel combined RF and B0 shim array for 3T brain imaging. Magn. Reson. Med. 75, 441\u2013451 (2016).","journal-title":"Magn. Reson. Med."},{"key":"411_CR50","doi-asserted-by":"publisher","unstructured":"Shin, D. et al. DeepRF: (v1.0) (Zenodo, 2021); https:\/\/doi.org\/10.5281\/zenodo.5529394","DOI":"10.5281\/zenodo.5529394"}],"container-title":["Nature Machine Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00411-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00411-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00411-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T20:41:16Z","timestamp":1670100076000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00411-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,16]]},"references-count":50,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["411"],"URL":"https:\/\/doi.org\/10.1038\/s42256-021-00411-1","relation":{},"ISSN":["2522-5839"],"issn-type":[{"value":"2522-5839","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,16]]},"assertion":[{"value":"26 April 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 October 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 November 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Two patents are disclosed (in the order of patent applicant, names of inventors, application number, status of application, specific aspect of manuscript covered in patent application). Seoul National University (D.S. and J.L.), US Patent No. 17\/168,274, patent pending; see the 'RF refinement module' section in the . Seoul National University (D.S. and J.L.), Korea Patent No. 10-2020-0106569, patent pending, see the 'RF refinement module' section in the .","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}