{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T20:44:57Z","timestamp":1783543497618,"version":"3.55.0"},"reference-count":51,"publisher":"Society of Exploration Geophysicists","issue":"4","license":[{"start":{"date-parts":[[2025,6,14]],"date-time":"2025-06-14T00:00:00Z","timestamp":1749859200000},"content-version":"vor","delay-in-days":158,"URL":"http:\/\/www.niso.org\/schemas\/ali\/1.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"crossref","award":["RGPIN-2021-02528"],"award-info":[{"award-number":["RGPIN-2021-02528"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"crossref","award":["RGPIN-2021-02528"],"award-info":[{"award-number":["RGPIN-2021-02528"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["library.seg.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,1,7]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>Geologic carbon storage (GCS) must be safe and profitable. To achieve these goals for gigaton-scale GCS operations, decision-making in the presence of uncertainty is required. Geophysical monitoring methods can inform such decisions, given their sensitivity to the spatiotemporal changes in the subsurface during and after injection. We investigate a novel framework for the optimal control of GCS operations using geophysical monitoring. We refer to this decision-making tool as \u201cgeophysical control\u201d and develop sequential decision-making models trained using digital twins of GCS operations and the corresponding geophysical monitoring signals. In particular, we obtain these models via deep reinforcement learning (DRL) and specifically focus on two types of uncertainty: geophysical noise and uncertainty in the subsurface petrophysical model. Our objective is to demonstrate how each source of stochasticity affects the decision-making process when one seeks to maximize profit while minimizing the risk of induced seismicity through an optimal policy that determines the annual target CO2 injection rate. We train a suite of DRL agents with different geophysical observations (surface time-lapse gravity, surface seismic amplitude-variation-with-offset (AVO), and combined gravity and AVO surveys), different signal-to-noise ratio levels, and with\/without petrophysical model uncertainties. A comparison of the learning behavior of these independent DRL agents shows that (1)\u00a0the DRL framework has the capacity to learn optimal CO2 injection policies; (2)\u00a0training performance degrades with increasing geophysical noise (especially more in the seismic AVO case); and (3)\u00a0the combination of AVO and gravity enhances decision-making, especially in the presence of geophysical noise. Our results show that the use of multigeophysical measurements and the incorporation of subsurface model uncertainties are critical in developing robust injection control agents using DRL.<\/jats:p>","DOI":"10.1190\/geo2024-0402.1","type":"journal-article","created":{"date-parts":[[2025,3,22]],"date-time":"2025-03-22T12:36:27Z","timestamp":1742646987000},"page":"H1-H14","update-policy":"https:\/\/doi.org\/10.1190\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Geophysical control of geologic carbon storage using deep reinforcement learning: Sensitivity to multigeophysical noise and to the uncertainty of digital twins"],"prefix":"10.1190","volume":"90","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5484-2981","authenticated-orcid":false,"given":"Kyubo","family":"Noh","sequence":"first","affiliation":[{"name":"University of Toronto 1 , Department of Earth Sciences, Toronto, Ontario, Canada. noh.kyubo@utoronto.ca (corresponding author)"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6435-6730","authenticated-orcid":false,"given":"Andrei","family":"Swidinsky","sequence":"additional","affiliation":[{"name":"University of Toronto 1 , Department of Earth Sciences, Toronto, Ontario, Canada. andrei.swidinsky@utoronto.ca"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"186","published-online":{"date-parts":[[2025,6,13]]},"reference":[{"key":"2025121212293550600_r1","doi-asserted-by":"crossref","DOI":"10.1037\/a0038591","article-title":"A taxonomy of endogenous and exogenous uncertainty in high-risk, high-impact contexts","volume":"100","author":"Alison","year":"2015","journal-title":"Journal of Applied Psychology","ISSN":"https:\/\/id.crossref.org\/issn\/0021-9010","issn-type":"print"},{"key":"2025121212293550600_r2","doi-asserted-by":"crossref","first-page":"5","DOI":"10.21314\/JOR.2001.041","article-title":"Optimal execution of portfolio transactions","volume":"3","author":"Almgren","year":"2001","journal-title":"Journal of Risk"},{"key":"2025121212293550600_r3","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijggc.2020.102956","article-title":"Time-lapse gravity monitoring of CO2 migration based on numerical modeling of a faulted storage complex","volume":"95","author":"Appriou","year":"2020","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"},{"key":"2025121212293550600_r4","volume-title":"The uncemented (soft) sand model in the rock physics handbook, section 5.5","author":"Avseth","year":"2010"},{"key":"2025121212293550600_r5","doi-asserted-by":"crossref","first-page":"1396","DOI":"10.1190\/1.1443207","article-title":"Seismic properties of pore fluids","volume":"57","author":"Batzle","year":"1992","journal-title":"Geophysics","ISSN":"https:\/\/id.crossref.org\/issn\/0016-8033","issn-type":"print"},{"key":"2025121212293550600_r200","first-page":"679","article-title":"A Markovian decision process","author":"Bellman","year":"1957","journal-title":"Journal of Mathematics and Mechanics"},{"key":"2025121212293550600_r6","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.ijggc.2012.01.001","article-title":"Impact-driven pressure management via targeted brine extraction \u2014 Conceptual studies of CO2 storage in saline formations","volume":"7","author":"Birkholzer","year":"2012","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"},{"key":"2025121212293550600_r7","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.petrol.2015.06.010","article-title":"Cross-well seismic and electromagnetic tomography for CO2 detection and monitoring in a saline aquifer","volume":"133","author":"B\u00f6hm","year":"2015","journal-title":"Journal of Petroleum Science and Engineering","ISSN":"https:\/\/id.crossref.org\/issn\/0920-4105","issn-type":"print"},{"key":"2025121212293550600_r8","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijggc.2024.104262","article-title":"Optimization of pressure management strategies for geological CO2 storage using surrogate model-based reinforcement learning","volume":"138","author":"Chen","year":"2024","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"},{"key":"2025121212293550600_r9","doi-asserted-by":"crossref","first-page":"56","DOI":"10.2118\/118926-PA","article-title":"Ensemble-based closed-loop optimization applied to Brugge field","volume":"13","author":"Chen","year":"2010","journal-title":"SPE Reservoir Evaluation & Engineering"},{"key":"2025121212293550600_r10","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijggc.2022.103717","article-title":"Improved geophysical monitoring of carbon sequestration through parameter linkage to reservoir modeling","volume":"119","author":"Commer","year":"2022","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"},{"key":"2025121212293550600_r11","volume-title":"Reinforcement learning for well location optimization","author":"Dawar","year":"2021"},{"key":"2025121212293550600_r12","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1038\/s41586-021-04301-9","article-title":"Magnetic control of tokamak plasmas through deep reinforcement learning","volume":"602","author":"Degrave","year":"2022","journal-title":"Nature"},{"key":"2025121212293550600_r13","doi-asserted-by":"crossref","first-page":"2277","DOI":"10.1016\/j.egypro.2009.01.296","article-title":"Recent advances in well-based monitoring of CO2 sequestration","volume":"1","author":"Freifeld","year":"2009","journal-title":"Energy Procedia","ISSN":"https:\/\/id.crossref.org\/issn\/1876-6102","issn-type":"print"},{"key":"2025121212293550600_r14","article-title":"A digital twin for geological carbon storage with controlled injectivity","author":"Gahlot","year":"2024"},{"issue":"2","key":"2025121212293550600_r15","doi-asserted-by":"crossref","first-page":"F13","DOI":"10.1190\/1.2181309","article-title":"Hysteresis effects studied by numerical simulations: Cyclic loading-unloading of a realistic sand model","volume":"71","author":"Garc\u00eda","year":"2006","journal-title":"Geophysics","ISSN":"https:\/\/id.crossref.org\/issn\/0016-8033","issn-type":"print"},{"key":"2025121212293550600_r16","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1190\/tle42110730.1","article-title":"President\u2019s page: Digital twins in the era of generative AI","volume":"42","author":"Herrmann","year":"2023","journal-title":"The Leading Edge"},{"key":"2025121212293550600_r17","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep reinforcement learning for autonomous driving: A survey","volume":"23","author":"Kiran","year":"2021","journal-title":"IEEE Transactions on Intelligent Transportation Systems"},{"key":"2025121212293550600_r18","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: A survey","volume":"32","author":"Kober","year":"2013","journal-title":"The International Journal of Robotics Research"},{"key":"2025121212293550600_r19","doi-asserted-by":"crossref","first-page":"1060","DOI":"10.1190\/1.3640530","article-title":"Understanding the applications and limitations of time-lapse gravity for reservoir monitoring","volume":"30","author":"Krahenbuhl","year":"2011","journal-title":"The Leading Edge"},{"key":"2025121212293550600_r20","doi-asserted-by":"crossref","first-page":"893","DOI":"10.1016\/j.reseneeco.2010.11.002","article-title":"Co-optimization of enhanced oil recovery and carbon sequestration","volume":"33","author":"Leach","year":"2011","journal-title":"Resource and Energy Economics"},{"key":"2025121212293550600_r21","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.eng.2022.06.011","article-title":"Bayesian optimization for field-scale geological carbon storage","volume":"18","author":"Lu","year":"2022","journal-title":"Engineering","ISSN":"https:\/\/id.crossref.org\/issn\/0013-7782","issn-type":"print"},{"key":"2025121212293550600_r22","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"2025121212293550600_r23","doi-asserted-by":"crossref","first-page":"3161","DOI":"10.5194\/gmd-15-3161-2022","article-title":"GSTools v1.3: A toolbox for geostatistical modeling in Python","volume":"15","author":"M\u00fcller","year":"2022","journal-title":"Geoscientific Model Development"},{"key":"2025121212293550600_r24","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1111\/1467-9868.00389","article-title":"Optimal dynamic treatment regimes","volume":"65","author":"Murphy","year":"2003","journal-title":"Journal of the Royal Statistical Society, Series B"},{"key":"2025121212293550600_r25","unstructured":"Nair\n              V.\n            \n            \n              Hinton\n              G. E.\n            \n          , 2010, Rectified linear units improve restricted Boltzmann machines: Proceedings of the 27th International Conference on Machine Learning, 807\u2013814."},{"key":"2025121212293550600_r26","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s10596-023-10255-w","article-title":"Multi-asset closed-loop reservoir management using deep reinforcement learning","volume":"28","author":"Nasir","year":"2024","journal-title":"Computational Geosciences"},{"key":"2025121212293550600_r27","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijggc.2024.104238","article-title":"Stochastic control of geological carbon storage operations using geophysical monitoring and deep reinforcement learning","volume":"138","author":"Noh","year":"2024","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"},{"key":"2025121212293550600_r28","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijggc.2021.103437","article-title":"Seismic monitoring of a small CO2 injection using a multi-well DAS array: Operations and initial results of Stage 3 of the CO2CRC Otway project","volume":"110","author":"Pevzner","year":"2021","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"},{"key":"2025121212293550600_r29","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.camwa.2020.05.014","article-title":"The open porous media flow reservoir simulator","volume":"81","author":"Rasmussen","year":"2021","journal-title":"Computers & Mathematics with Applications"},{"key":"2025121212293550600_r30","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.ces.2016.01.021","article-title":"Numerical simulation and optimization of CO2 utilization for enhanced oil recovery from depleted reservoirs","volume":"144","author":"Safi","year":"2016","journal-title":"Chemical Engineering Science","ISSN":"https:\/\/id.crossref.org\/issn\/0009-2509","issn-type":"print"},{"key":"2025121212293550600_r31","doi-asserted-by":"crossref","DOI":"10.1002\/9780470316849","volume-title":"Multivariate density estimation: Theory, practice, and visualization","author":"Scott","year":"1992"},{"key":"2025121212293550600_r32","doi-asserted-by":"crossref","DOI":"10.1016\/j.jconhyd.2021.103835","article-title":"Physics-informed deep learning for prediction of CO2 storage site response","volume":"241","author":"Shokouhi","year":"2021","journal-title":"Journal of Contaminant Hydrology","ISSN":"https:\/\/id.crossref.org\/issn\/0169-7722","issn-type":"print"},{"key":"2025121212293550600_r33","doi-asserted-by":"crossref","DOI":"10.1016\/j.apenergy.2020.115660","article-title":"Optimal carbon storage reservoir management through deep reinforcement learning","volume":"278","author":"Sun","year":"2020","journal-title":"Applied Energy","ISSN":"https:\/\/id.crossref.org\/issn\/0306-2619","issn-type":"print"},{"key":"2025121212293550600_r34","volume-title":"Reinforcement learning: An introduction","author":"Sutton","year":"2018"},{"key":"2025121212293550600_r35","doi-asserted-by":"crossref","first-page":"2879","DOI":"10.1016\/j.egypro.2014.11.311","article-title":"Coupled dynamic flow and geomechanical simulations for an integrated assessment of CO2 storage impacts in a saline aquifer","volume":"63","author":"Tillner","year":"2014","journal-title":"Energy Procedia","ISSN":"https:\/\/id.crossref.org\/issn\/1876-6102","issn-type":"print"},{"key":"2025121212293550600_r36","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1111\/1365-2478.13257","article-title":"Deep learning multiphysics network for imaging CO2 saturation and estimating uncertainty in geological carbon storage","volume":"72","author":"Um","year":"2023","journal-title":"Geophysical Prospecting","ISSN":"https:\/\/id.crossref.org\/issn\/0016-8025","issn-type":"print"},{"key":"2025121212293550600_r37","doi-asserted-by":"crossref","first-page":"938","DOI":"10.1002\/2017RG000566","article-title":"Geophysics from terrestrial time-variable gravity measurements","volume":"55","author":"Van Camp","year":"2017","journal-title":"Reviews of Geophysics","ISSN":"https:\/\/id.crossref.org\/issn\/8755-1209","issn-type":"print"},{"key":"2025121212293550600_r38","doi-asserted-by":"crossref","unstructured":"Van Hasselt\n              H.\n            \n            \n              Guez\n              A.\n            \n            \n              Silver\n              D.\n            \n          , 2016, Deep reinforcement learning with double Q-learning: Proceedings of the 30th AAAI Conference on Artificial Intelligence.","DOI":"10.1609\/aaai.v30i1.10295"},{"issue":"5","key":"2025121212293550600_r39","doi-asserted-by":"crossref","first-page":"Z102","DOI":"10.1190\/2011-0926-GEODIS.6","article-title":"Microseismic monitoring and geomechanical modeling of CO2 storage in subsurface reservoirs","volume":"76","author":"Verdon","year":"2011","journal-title":"Geophysics","ISSN":"https:\/\/id.crossref.org\/issn\/0016-8033","issn-type":"print"},{"key":"2025121212293550600_r40","doi-asserted-by":"crossref","first-page":"8654","DOI":"10.1109\/JIOT.2020.2995617","article-title":"The internet of things in the oil and gas industry: A systematic review","volume":"7","author":"Wanasinghe","year":"2020","journal-title":"IEEE Internet of Things Journal"},{"key":"2025121212293550600_r41","doi-asserted-by":"crossref","DOI":"10.1016\/j.advwatres.2022.104180","article-title":"U-FNO \u2014 An enhanced Fourier neural operator-based deep-learning model for multiphase flow","volume":"163","author":"Wen","year":"2022","journal-title":"Advances in Water Resources","ISSN":"https:\/\/id.crossref.org\/issn\/0309-1708","issn-type":"print"},{"key":"2025121212293550600_r42","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1017\/9781316480724.016","volume-title":"Geophysics and geosequestration","author":"White","year":"2019"},{"key":"2025121212293550600_r43","first-page":"4886","article-title":"Closing the loop: 4D seismic constraints on CO2 flow simulations from the Aquistore CO2 storage site","volume":"114","author":"White","year":"2017","journal-title":"Energy Procedia","ISSN":"https:\/\/id.crossref.org\/issn\/1876-6102","issn-type":"print"},{"issue":"2","key":"2025121212293550600_r44","doi-asserted-by":"crossref","first-page":"F21","DOI":"10.1190\/1.2187734","article-title":"Calculation of CO2 acoustic properties using Batzle-Wang equations","volume":"71","author":"Xu","year":"2006","journal-title":"Geophysics","ISSN":"https:\/\/id.crossref.org\/issn\/0016-8033","issn-type":"print"},{"key":"2025121212293550600_r45","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.ijggc.2015.06.016","article-title":"InSAR monitoring of ground deformation due to CO2 injection at an enhanced oil recovery site, West Texas","volume":"41","author":"Yang","year":"2015","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"},{"key":"2025121212293550600_r46","doi-asserted-by":"crossref","DOI":"10.1016\/j.softx.2023.101567","article-title":"rockphypy: An extensive Python library for rock physics modeling","volume":"24","author":"Yu","year":"2023","journal-title":"SoftwareX"},{"key":"2025121212293550600_r48","doi-asserted-by":"crossref","DOI":"10.1016\/j.jngse.2022.104814","article-title":"Extension of CO2 storage life in the Sleipner CCS project by reservoir pressure management","volume":"108","author":"Zhang","year":"2022","journal-title":"Journal of Natural Gas Science and Engineering"},{"key":"2025121212293550600_r47","article-title":"Deep reinforcement learning for cost-effective medical diagnosis","author":"Yu","year":"2023"},{"key":"2025121212293550600_r49","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1109\/TPWRS.2021.3105418","article-title":"Robust scheduling of virtual power plant under exogenous and endogenous uncertainties","volume":"37","author":"Zhang","year":"2021","journal-title":"IEEE Transactions on Power Systems","ISSN":"https:\/\/id.crossref.org\/issn\/0885-8950","issn-type":"print"},{"key":"2025121212293550600_r50","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijggc.2021.103385","article-title":"Geologic CO2 storage optimization under geomechanical risk using coupled-physics models","volume":"110","author":"Zheng","year":"2021","journal-title":"International Journal of Greenhouse Gas Control","ISSN":"https:\/\/id.crossref.org\/issn\/1750-5836","issn-type":"print"}],"container-title":["Geophysics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.geoscienceworld.org\/seg\/geophysics\/article-pdf\/90\/4\/H1\/7244803\/geo-2024-0402.1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.geoscienceworld.org\/seg\/geophysics\/article-pdf\/90\/4\/H1\/7244803\/geo-2024-0402.1.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T17:29:48Z","timestamp":1765560588000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.geoscienceworld.org\/geophysics\/article\/90\/4\/H1\/657239\/Geophysical-control-of-geologic-carbon-storage"},"secondary":[{"URL":"https:\/\/pubs.geoscienceworld.org\/geophysics\/article\/90\/4\/h1\/657239\/geophysical-control-of-geologic-carbon-storage","label":"geoscienceworld"}]},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,7]]},"references-count":51,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,1,7]]}},"URL":"https:\/\/doi.org\/10.1190\/geo2024-0402.1","relation":{},"ISSN":["0016-8033","1942-2156"],"issn-type":[{"value":"0016-8033","type":"print"},{"value":"1942-2156","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,7]]},"assertion":[{"value":"2024-06-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-18","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-28","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}