{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T16:33:37Z","timestamp":1774888417730,"version":"3.50.1"},"reference-count":44,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T00:00:00Z","timestamp":1723161600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In this work, we explore the application of deep reinforcement learning (DRL) to algorithmic trading. While algorithmic trading is focused on using computer algorithms to automate a predefined trading strategy, in this work, we train a Double Deep Q-Network (DDQN) agent to learn its own optimal trading policy, with the goal of maximising returns whilst managing risk. In this study, we extended our approach by augmenting the Markov Decision Process (MDP) states with sentiment analysis of financial statements, through which the agent achieved up to a 70% increase in the cumulative reward over the testing period and an increase in the Calmar ratio from 0.9 to 1.3. The experimental results also showed that the DDQN agent\u2019s trading strategy was able to consistently outperform the benchmark set by the buy-and-hold strategy. Additionally, we further investigated the impact of the length of the window of past market data that the agent considers when deciding on the best trading action to take. The results of this study have validated DRL\u2019s ability to find effective solutions and its importance in studying the behaviour of agents in markets. This work serves to provide future researchers with a foundation to develop more advanced and adaptive DRL-based trading systems.<\/jats:p>","DOI":"10.3390\/info15080473","type":"journal-article","created":{"date-parts":[[2024,8,10]],"date-time":"2024-08-10T06:25:24Z","timestamp":1723271124000},"page":"473","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Algorithmic Trading Using Double Deep Q-Networks and Sentiment Analysis"],"prefix":"10.3390","volume":"15","author":[{"given":"Leon","family":"Tabaro","sequence":"first","affiliation":[{"name":"Department of Computer Science, Loughborough University, Epinal Way, Loughborough LE11 3TU, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5237-3050","authenticated-orcid":false,"given":"Jean Marie Vianney","family":"Kinani","sequence":"additional","affiliation":[{"name":"Departamento de Mecatr\u00f3nica, Instituto Polit\u00e9cnico Nacional\u2014UPIIH, Pachuca 07738, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8436-3025","authenticated-orcid":false,"given":"Alberto Jorge","family":"Rosales-Silva","sequence":"additional","affiliation":[{"name":"Secci\u00f3n de Estudios de Posgrado e Investigaci\u00f3n, Instituto Polit\u00e9cnico Nacional\u2014ESIME Zacatenco, Mexico City 07738, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1666-9924","authenticated-orcid":false,"given":"Julio C\u00e9sar","family":"Salgado-Ram\u00edrez","sequence":"additional","affiliation":[{"name":"Ingenier\u00eda Biom\u00e9dica, Universidad Polit\u00e9cnica de Pachuca, Zempoala 43830, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8665-4096","authenticated-orcid":false,"given":"Dante","family":"M\u00fajica-Vargas","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tecnol\u00f3gico Nacional de M\u00e9xico\/CENIDET, Interior Internado Palmira S\/N, Palmira, Cuernavaca 62490, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ponciano Jorge","family":"Escamilla-Ambrosio","sequence":"additional","affiliation":[{"name":"Centro de Investigaci\u00f3n en Computaci\u00f3n, Instituto Polit\u00e9cnico Nacional, Mexico City 07700, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2195-970X","authenticated-orcid":false,"given":"Eduardo","family":"Ramos-D\u00edaz","sequence":"additional","affiliation":[{"name":"Ingenier\u00eda en Sistemas Electr\u00f3nicos y de Telecomunicaciones, Universidad Aut\u00f3noma de la Ciudad de M\u00e9xico, Mexico City 09790, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,9]]},"reference":[{"key":"ref_1","unstructured":"Chan, E. (2009). Quantitative Trading: How to Build Your Own Algorithmic Trading Business, Wiley Trading; Wiley."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chan, E. (2013). Algorithmic Trading: Winning Strategies and Their Rationale, Wiley Trading; Wiley.","DOI":"10.1002\/9781118676998"},{"key":"ref_3","unstructured":"Zimmermann, H. (2021). Intraday Trading with Neural Networks and Deep Reinforcement Learning, Imperial College London."},{"key":"ref_4","unstructured":"Maven (2024, May 17). Machine Learning in Algorithmic Trading, Maven Securities. Available online: https:\/\/www.mavensecurities.com\/machine-learning-in-algorithmic-trading\/."},{"key":"ref_5","unstructured":"Spooner, T. (2021). Algorithmic Trading and Reinforcement Learning: Robust Methodologies for AI in Finance. [Ph.D. Thesis, The University of Liverpool Repository]. Available online: https:\/\/livrepository.liverpool.ac.uk\/3130139\/."},{"key":"ref_6","first-page":"679","article-title":"A Markovian Decision Process","volume":"6","author":"Bellman","year":"1957","journal-title":"J. Math. Mech."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"van Hasselt, H., Guez, A., and Silver, D. (2016). Deep reinforcement learning with double Q-learning. arXiv.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_8","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2015). Playing Atari with deep reinforcement learning. arXiv."},{"key":"ref_9","unstructured":"Zejnullahu, F., Moser, M., and Osterrieder, J. (2022). Applications of reinforcement learning in Finance\u2014Trading with a double deep Q-Network. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Aldridge, I. (2013). High-Frequency Trading: A Practical Guide to Algorithmic Strategies and Trading Systems, John Wiley & Sons.","DOI":"10.1002\/9781119203803"},{"key":"ref_11","unstructured":"Savcenko, K. (2022). The \u2018A\u2019 Factor: The Role of Algorithmic Trading during an Energy Crisis, S&P Global Commodity Insights. Available online: https:\/\/www.spglobal.com\/commodityinsights\/en\/market-insights\/blogs\/electric-power\/110322-algorithm-trading-europe-energy-crisis."},{"key":"ref_12","unstructured":"Fischer, T.G. (2018). Reinforcement Learning in Financial Markets\u2014A Survey, Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg, Institute for Economics. FAU Discussion Papers in Economics; No. 12\/2018."},{"key":"ref_13","unstructured":"Neuneier, R. (2024, August 01). Optimal asset allocation using adaptive dynamic programming. Advances in Neural Information Processing Systems, Available online: https:\/\/proceedings.neurips.cc\/paper\/1995\/hash\/3a15c7d0bbe60300a39f76f8a5ba6896-Abstract.html."},{"key":"ref_14","unstructured":"Jin, O., and El-Saawy, H. (2016). Portfolio Management Using Reinforcement Learning, Stanford University. Working Paper."},{"key":"ref_15","unstructured":"Liang, Z., Chen, H., Zhu, J., Jiang, K., and Li, Y. (2018). Adversarial deep reinforcement learning in portfolio management. arXiv."},{"key":"ref_16","unstructured":"Ding, X., Zhang, Y., Liu, T., and Duan, J. (2015, January 25\u201331). Deep learning for event-driven stock prediction. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_17","first-page":"146","article-title":"The selection of reinforcement learning state and value function applied to portfolio optimization","volume":"48","author":"Zhu","year":"2020","journal-title":"J. Fuzhou Univ. (Nat. Sci. Ed.)"},{"key":"ref_18","first-page":"23","article-title":"An application of reinforcement learning based approach to stock trading","volume":"3","author":"Dai","year":"2021","journal-title":"Bus. Manag."},{"key":"ref_19","unstructured":"Ning, B., Lin, F.H.T., and Jaimungal, S. (2018). Double deep Q-learning for optimal execution. arXiv."},{"key":"ref_20","unstructured":"(2024, February 02). Machine Learning Trading. Trading with Deep Reinforcement Learning. Dr Thomas Starke (2020) YouTube. Available online: https:\/\/www.youtube.com\/watch?v=H-c49jQxGbs."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Nevmyvaka, Y., Feng, Y., and Kearns, M. (2006, January 25\u201329). Reinforcement learning for optimized trade execution. Proceedings of the 23rd International Conference on Machine Learning, ACM, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143929"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/S2212-5671(12)00122-0","article-title":"Testing different reinforcement learning configurations for financial trading: Introduction and applications","volume":"3","author":"Bertoluzzo","year":"2012","journal-title":"Procedia Econ. Financ."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.dss.2014.04.011","article-title":"Intelligent trading of seasonal effects: A decision support algorithm based on reinforcement learning","volume":"64","author":"Eilers","year":"2014","journal-title":"Decis. Support Syst."},{"key":"ref_24","unstructured":"Sherstov, A.A., and Stone, P. (2004). Three automated stock-trading agents: A comparative study. Proceeedings of the International Workshop on Agent-Mediated Electronic Commerce, Springer."},{"key":"ref_25","unstructured":"Kaur, S. (2017). Algorithmic Trading Using Sentiment Analysis and Reinforcement Learning, Stanford University. Working Paper."},{"key":"ref_26","first-page":"75","article-title":"Deep reinforcement learning stock algorithm trading system application","volume":"16","author":"Rong","year":"2020","journal-title":"J. Comput. Knowl. Technol."},{"key":"ref_27","unstructured":"Li, Y., Zhou, P., Li, F., and Yang, X. (2021). An improved reinforcement learning model based on sentiment analysis. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Pardo, R. (2011). The Evaluation and Optimization of Trading Strategies, John Wiley & Sons.","DOI":"10.1002\/9781119196969"},{"key":"ref_29","unstructured":"Hu, G. (2023). Advancing algorithmic trading: A multi-technique enhancement of deep Q-network models. arXiv."},{"key":"ref_30","unstructured":"Tesla, Inc. (TSLA) (2024, April 02). Stock Historical Prices Data\u2014Yahoo Finance\u2014finance.yahoo.com. Available online: https:\/\/finance.yahoo.com\/quote\/TSLA\/history?p=TSLA."},{"key":"ref_31","unstructured":"(2024, May 02). SEC.gov\u2014EDGAR Full Text Search\u2014sec.gov, Available online: https:\/\/www.sec.gov\/edgar\/search\/#\/q=(Annual%2520report)&dateRange=all&ciks=0001318605&entityName=Tesla%252C%2520Inc.%2520(TSLA)%2520(CIK%25200001318605)."},{"key":"ref_32","unstructured":"(2024, May 02). Marketing Communications: Web\/\/University of Notre Dame Loughran-McDonald master Dictionary W\/Sentiment Word Lists\/\/Software Repository for Accounting and Finance\/\/University of Notre Dame, Software Repository for Accounting and Finance. Available online: https:\/\/sraf.nd.edu\/loughranmcdonald-master-dictionary\/."},{"key":"ref_33","unstructured":"(2024, May 02). Loughran-McDonald Master Dictionary w\/Sentiment Word Lists\/\/Software Repository for Accounting and Finance\/\/University of Notre Dame\u2014sraf.nd.edu, Available online: https:\/\/sraf.nd.edu\/loughranmcdonald-master-dictionary\/."},{"key":"ref_34","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). OpenAI Gym. arXiv."},{"key":"ref_35","unstructured":"Carapu\u00e7o, J.M.B. (2024, February 12). Reinforcement Learning Applied to Forex Trading, Scribd. Available online: https:\/\/www.scribd.com\/document\/449849827\/Corrected-Thesis-JoaoMaria67923."},{"key":"ref_36","first-page":"40","article-title":"Calmar ratio: A smoother tool","volume":"20","author":"Young","year":"1991","journal-title":"Futures"},{"key":"ref_37","unstructured":"(2024, May 02). Edgar Filing Documents for 0001564590-17-015705, Available online: https:\/\/www.sec.gov\/Archives\/edgar\/data\/1318605\/000156459017015705\/0001564590-17-015705-index.htm."},{"key":"ref_38","unstructured":"(2024, May 02). Edgar Filing Documents for 0001564590-15-001031, Available online: https:\/\/www.sec.gov\/Archives\/edgar\/data\/1318605\/000156459015001031\/0001564590-15-001031-index.htm."},{"key":"ref_39","unstructured":"Murphy, J.J. (1999). Technical Analysis of the Financial Markets: A Comprehensive Guide to Trading Methods and Applications, Penguin Publishing Group. Available online: https:\/\/www.google.com\/books\/edition\/_\/5zhXEqdr_IcC?hl=en&gbpv=0."},{"key":"ref_40","unstructured":"Wilder, J.W. (1978). New Concepts in Technical Trading Systems, Trend Research. Available online: https:\/\/archive.org\/details\/newconceptsintec00wild\/page\/n151\/mode\/2up."},{"key":"ref_41","unstructured":"Jahn, M. (2024, July 24). What Is the Haurlan Index? Investopedia. Available online: https:\/\/www.investopedia.com\/terms\/h\/haurlanindex.asp#:~:text=The%20Haurlan%20Index%20was%20developed,the%20New%20York%20Stock%20Exchange."},{"key":"ref_42","unstructured":"Ushman, D. (2024, July 24). What Is the SMA Indicator (Simple Moving Average). Available online: https:\/\/trendspider.com\/learning-center\/what-is-the-sma-indicator-simple-moving-average\/."},{"key":"ref_43","first-page":"18","article-title":"Balance Of Power","volume":"19","author":"Livshin","year":"2001","journal-title":"Tech. Anal. Stock. Commod."},{"key":"ref_44","unstructured":"Mitchell, C. (2024, July 24). Aroon Oscillator: Definition, Calculation Formula, Trade Signals, Investopedia. Available online: https:\/\/www.investopedia.com\/terms\/a\/aroonoscillator.asp."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/8\/473\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:33:00Z","timestamp":1760110380000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/8\/473"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,9]]},"references-count":44,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["info15080473"],"URL":"https:\/\/doi.org\/10.3390\/info15080473","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,9]]}}}