none 10.1190/geo2024-0402.1 Society of Exploration Geophysicists Society of Exploration Geophysicists 186 173882124 5981 2025121212293550600 10.1190 2025-12-12T17:29:48Z 2025-03-22T12:36:27Z 6 Geophysics 0016-8033 1942-2156 01 07 2025 90 4 Kyubo Noh University of Toronto 1 , Department of Earth Sciences, Toronto, Ontario, Canada. noh.kyubo@utoronto.ca (corresponding author) http://orcid.org/0000-0001-5484-2981 Andrei Swidinsky University of Toronto 1 , Department of Earth Sciences, Toronto, Ontario, Canada. andrei.swidinsky@utoronto.ca http://orcid.org/0000-0002-6435-6730 ABSTRACT Geologic carbon storage (GCS) must be safe and profitable. To achieve these goals for gigaton-scale GCS operations, decision-making in the presence of uncertainty is required. Geophysical monitoring methods can inform such decisions, given their sensitivity to the spatiotemporal changes in the subsurface during and after injection. We investigate a novel framework for the optimal control of GCS operations using geophysical monitoring. We refer to this decision-making tool as “geophysical control” and develop sequential decision-making models trained using digital twins of GCS operations and the corresponding geophysical monitoring signals. In particular, we obtain these models via deep reinforcement learning (DRL) and specifically focus on two types of uncertainty: geophysical noise and uncertainty in the subsurface petrophysical model. Our objective is to demonstrate how each source of stochasticity affects the decision-making process when one seeks to maximize profit while minimizing the risk of induced seismicity through an optimal policy that determines the annual target CO2 injection rate. We train a suite of DRL agents with different geophysical observations (surface time-lapse gravity, surface seismic amplitude-variation-with-offset (AVO), and combined gravity and AVO surveys), different signal-to-noise ratio levels, and with/without petrophysical model uncertainties. A comparison of the learning behavior of these independent DRL agents shows that (1) the DRL framework has the capacity to learn optimal CO2 injection policies; (2) training performance degrades with increasing geophysical noise (especially more in the seismic AVO case); and (3) the combination of AVO and gravity enhances decision-making, especially in the presence of geophysical noise. Our results show that the use of multigeophysical measurements and the incorporation of subsurface model uncertainties are critical in developing robust injection control agents using DRL. 06 13 2025 01 07 2025 H1 H14 NSERC 501100000038 RGPIN-2021-02528 http://www.niso.org/schemas/ali/1.0/ 1 10.1190/crossmark-policy library.seg.org true 2024-06-18 2025-02-18 2025-02-28 2025-06-13 NSERC http://dx.doi.org/10.13039/501100000038 RGPIN-2021-02528 10.1190/geo2024-0402.1 https://pubs.geoscienceworld.org/geophysics/article/90/4/H1/657239/Geophysical-control-of-geologic-carbon-storage https://pubs.geoscienceworld.org/seg/geophysics/article-pdf/90/4/H1/7244803/geo-2024-0402.1.pdf https://pubs.geoscienceworld.org/seg/geophysics/article-pdf/90/4/H1/7244803/geo-2024-0402.1.pdf https://pubs.geoscienceworld.org/geophysics/article/90/4/h1/657239/geophysical-control-of-geologic-carbon-storage 0021-9010 Journal of Applied Psychology Alison 100 2015 10.1037/a0038591 A taxonomy of endogenous and exogenous uncertainty in high-risk, high-impact contexts Journal of Risk Almgren 3 5 2001 10.21314/JOR.2001.041 Optimal execution of portfolio transactions 1750-5836 International Journal of Greenhouse Gas Control Appriou 95 2020 10.1016/j.ijggc.2020.102956 Time-lapse gravity monitoring of CO2 migration based on numerical modeling of a faulted storage complex The uncemented (soft) sand model in the rock physics handbook, section 5.5 Avseth 2010 0016-8033 Geophysics Batzle 57 1396 1992 10.1190/1.1443207 Seismic properties of pore fluids Journal of Mathematics and Mechanics Bellman 679 1957 A Markovian decision process 1750-5836 International Journal of Greenhouse Gas Control Birkholzer 7 168 2012 10.1016/j.ijggc.2012.01.001 Impact-driven pressure management via targeted brine extraction — Conceptual studies of CO2 storage in saline formations 0920-4105 Journal of Petroleum Science and Engineering Böhm 133 245 2015 10.1016/j.petrol.2015.06.010 Cross-well seismic and electromagnetic tomography for CO2 detection and monitoring in a saline aquifer 1750-5836 International Journal of Greenhouse Gas Control Chen 138 2024 10.1016/j.ijggc.2024.104262 Optimization of pressure management strategies for geological CO2 storage using surrogate model-based reinforcement learning SPE Reservoir Evaluation & Engineering Chen 13 56 2010 10.2118/118926-PA Ensemble-based closed-loop optimization applied to Brugge field 1750-5836 International Journal of Greenhouse Gas Control Commer 119 2022 10.1016/j.ijggc.2022.103717 Improved geophysical monitoring of carbon sequestration through parameter linkage to reservoir modeling Reinforcement learning for well location optimization Dawar 2021 Nature Degrave 602 414 2022 10.1038/s41586-021-04301-9 Magnetic control of tokamak plasmas through deep reinforcement learning 1876-6102 Energy Procedia Freifeld 1 2277 2009 10.1016/j.egypro.2009.01.296 Recent advances in well-based monitoring of CO2 sequestration Gahlot 2024 A digital twin for geological carbon storage with controlled injectivity 0016-8033 Geophysics García 71 2 F13 2006 10.1190/1.2181309 Hysteresis effects studied by numerical simulations: Cyclic loading-unloading of a realistic sand model The Leading Edge Herrmann 42 730 2023 10.1190/tle42110730.1 President’s page: Digital twins in the era of generative AI IEEE Transactions on Intelligent Transportation Systems Kiran 23 4909 2021 10.1109/TITS.2021.3054625 Deep reinforcement learning for autonomous driving: A survey The International Journal of Robotics Research Kober 32 1238 2013 10.1177/0278364913495721 Reinforcement learning in robotics: A survey The Leading Edge Krahenbuhl 30 1060 2011 10.1190/1.3640530 Understanding the applications and limitations of time-lapse gravity for reservoir monitoring Resource and Energy Economics Leach 33 893 2011 10.1016/j.reseneeco.2010.11.002 Co-optimization of enhanced oil recovery and carbon sequestration 0013-7782 Engineering Lu 18 96 2022 10.1016/j.eng.2022.06.011 Bayesian optimization for field-scale geological carbon storage Nature Mnih 518 529 2015 10.1038/nature14236 Human-level control through deep reinforcement learning Geoscientific Model Development Müller 15 3161 2022 10.5194/gmd-15-3161-2022 GSTools v1.3: A toolbox for geostatistical modeling in Python Journal of the Royal Statistical Society, Series B Murphy 65 331 2003 10.1111/1467-9868.00389 Optimal dynamic treatment regimes Nair V. Hinton G. E. , 2010, Rectified linear units improve restricted Boltzmann machines: Proceedings of the 27th International Conference on Machine Learning, 807–814. Computational Geosciences Nasir 28 23 2024 10.1007/s10596-023-10255-w Multi-asset closed-loop reservoir management using deep reinforcement learning 1750-5836 International Journal of Greenhouse Gas Control Noh 138 2024 10.1016/j.ijggc.2024.104238 Stochastic control of geological carbon storage operations using geophysical monitoring and deep reinforcement learning 1750-5836 International Journal of Greenhouse Gas Control Pevzner 110 2021 10.1016/j.ijggc.2021.103437 Seismic monitoring of a small CO2 injection using a multi-well DAS array: Operations and initial results of Stage 3 of the CO2CRC Otway project Computers & Mathematics with Applications Rasmussen 81 159 2021 10.1016/j.camwa.2020.05.014 The open porous media flow reservoir simulator 0009-2509 Chemical Engineering Science Safi 144 30 2016 10.1016/j.ces.2016.01.021 Numerical simulation and optimization of CO2 utilization for enhanced oil recovery from depleted reservoirs Multivariate density estimation: Theory, practice, and visualization Scott 1992 10.1002/9780470316849 0169-7722 Journal of Contaminant Hydrology Shokouhi 241 2021 10.1016/j.jconhyd.2021.103835 Physics-informed deep learning for prediction of CO2 storage site response 0306-2619 Applied Energy Sun 278 2020 10.1016/j.apenergy.2020.115660 Optimal carbon storage reservoir management through deep reinforcement learning Reinforcement learning: An introduction Sutton 2018 1876-6102 Energy Procedia Tillner 63 2879 2014 10.1016/j.egypro.2014.11.311 Coupled dynamic flow and geomechanical simulations for an integrated assessment of CO2 storage impacts in a saline aquifer 0016-8025 Geophysical Prospecting Um 72 183 2023 10.1111/1365-2478.13257 Deep learning multiphysics network for imaging CO2 saturation and estimating uncertainty in geological carbon storage 8755-1209 Reviews of Geophysics Van Camp 55 938 2017 10.1002/2017RG000566 Geophysics from terrestrial time-variable gravity measurements 10.1609/aaai.v30i1.10295 Van Hasselt H. Guez A. Silver D. , 2016, Deep reinforcement learning with double Q-learning: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 0016-8033 Geophysics Verdon 76 5 Z102 2011 10.1190/2011-0926-GEODIS.6 Microseismic monitoring and geomechanical modeling of CO2 storage in subsurface reservoirs IEEE Internet of Things Journal Wanasinghe 7 8654 2020 10.1109/JIOT.2020.2995617 The internet of things in the oil and gas industry: A systematic review 0309-1708 Advances in Water Resources Wen 163 2022 10.1016/j.advwatres.2022.104180 U-FNO — An enhanced Fourier neural operator-based deep-learning model for multiphase flow Geophysics and geosequestration White 257 2019 10.1017/9781316480724.016 1876-6102 Energy Procedia White 114 4886 2017 Closing the loop: 4D seismic constraints on CO2 flow simulations from the Aquistore CO2 storage site 0016-8033 Geophysics Xu 71 2 F21 2006 10.1190/1.2187734 Calculation of CO2 acoustic properties using Batzle-Wang equations 1750-5836 International Journal of Greenhouse Gas Control Yang 41 20 2015 10.1016/j.ijggc.2015.06.016 InSAR monitoring of ground deformation due to CO2 injection at an enhanced oil recovery site, West Texas SoftwareX Yu 24 2023 10.1016/j.softx.2023.101567 rockphypy: An extensive Python library for rock physics modeling Journal of Natural Gas Science and Engineering Zhang 108 2022 10.1016/j.jngse.2022.104814 Extension of CO2 storage life in the Sleipner CCS project by reservoir pressure management Yu 2023 Deep reinforcement learning for cost-effective medical diagnosis 0885-8950 IEEE Transactions on Power Systems Zhang 37 1311 2021 10.1109/TPWRS.2021.3105418 Robust scheduling of virtual power plant under exogenous and endogenous uncertainties 1750-5836 International Journal of Greenhouse Gas Control Zheng 110 2021 10.1016/j.ijggc.2021.103385 Geologic CO2 storage optimization under geomechanical risk using coupled-physics models