{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T14:57:02Z","timestamp":1775833022478,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T00:00:00Z","timestamp":1762300800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Firdaous Khemlichi"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Most reinforcement learning (RL) methods for portfolio optimization remain limited to single markets and a single algorithmic paradigm, which restricts their adaptability to regime shifts and heterogeneous conditions. This paper introduces a generalized version of the Modular Portfolio Learning System (MPLS), extending beyond its initial PPO backbone to integrate four RL algorithms: Proximal Policy Optimization (PPO), Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Soft Actor-Critic (SAC). Building on its modular design, MPLS leverages specialized components for sentiment analysis, volatility forecasting, and structural dependency modeling, whose signals are fused within an attention-based decision framework. Unlike prior approaches, MPLS is evaluated independently on three major equity indices (S&amp;P 500, DAX 30, and FTSE 100) across diverse regimes including stable, crisis, recovery, and sideways phases. Experimental results show that MPLS consistently achieved higher Sharpe ratios\u2014typically +40\u201370% over Minimum Variance Portfolio (MVP) and Risk Parity (RP)\u2014while limiting drawdowns and Conditional Value-at-Risk (CVaR) during stress periods such as the COVID-19 crash. Turnover levels remained moderate, confirming cost-awareness. Ablation and variance analyses highlight the distinct contribution of each module and the robustness of the framework. Overall, MPLS represents a modular, resilient, and practically relevant framework for risk-aware portfolio optimization.<\/jats:p>","DOI":"10.3390\/info16110961","type":"journal-article","created":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T17:06:06Z","timestamp":1762362366000},"page":"961","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Modular Reinforcement Learning for Multi-Market Portfolio Optimization"],"prefix":"10.3390","volume":"16","author":[{"given":"Firdaous","family":"Khemlichi","sequence":"first","affiliation":[{"name":"Laboratory of Intelligent Systems, Georesources & Renewable Energies (SIGER), University Sidi Mohamed Ben Abdellah, Fez 30000, Morocco"}]},{"given":"Youness","family":"Idrissi Khamlichi","sequence":"additional","affiliation":[{"name":"Laboratory of Intelligent Systems, Georesources & Renewable Energies (SIGER), University Sidi Mohamed Ben Abdellah, Fez 30000, Morocco"}]},{"given":"Safae","family":"Elhaj Ben Ali","sequence":"additional","affiliation":[{"name":"Laboratory of Intelligent Systems, Georesources & Renewable Energies (SIGER), University Sidi Mohamed Ben Abdellah, Fez 30000, Morocco"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1057\/s41599-025-04850-8","article-title":"AI integration in financial services: A systematic review of trends and regulatory challenges","volume":"12","year":"2025","journal-title":"Humanit. Soc. Sci. Commun."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sutiene, K., Schwendner, P., Sipos, C., Lorenzo, L., Mirchev, M., Lameski, P., and Cerneviciene, J. (2024). Enhancing Portfolio Management Using Artificial Intelligence: Literature Review. Front. Artif. Intell., 7.","DOI":"10.3389\/frai.2024.1371502"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1429","DOI":"10.1017\/S0022109017000461","article-title":"Regulatory Sanctions and Reputational Damage in Financial Markets","volume":"52","author":"Armour","year":"2017","journal-title":"J. Financ. Quant. Anal."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"13","DOI":"10.2307\/1924119","article-title":"The Valuation of Risk Assets and the Selection of Risky Investments in Stock Portfolios and Capital Budgets","volume":"47","author":"Lintner","year":"1965","journal-title":"Rev. Econ. Stat."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/0304-405X(93)90023-5","article-title":"Common risk factors in the returns on stocks and bonds","volume":"33","author":"Fama","year":"1993","journal-title":"J. Financ. Econ."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_7","unstructured":"Jiang, Z., Xu, D., and Liang, J. (2017). A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"875","DOI":"10.1109\/72.935097","article-title":"Learning to trade via direct reinforcement","volume":"12","author":"Moody","year":"2001","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_9","unstructured":"Doshi-Velez, F., and Kim, B. (2017). Towards a Rigorous Science of Interpretable Machine Learning. arXiv."},{"key":"ref_10","first-page":"1959","article-title":"MPLS: A Modular Portfolio Learning System for Adaptive Portfolio Optimization","volume":"12","author":"Khemlichi","year":"2025","journal-title":"Math. Model. Eng. Probl."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yan, Y., Zhang, C., An, Y., and Zhang, B. (2025). A Deep-Reinforcement-Learning-Based Multi-Source Information Fusion Portfolio Management Approach via Sector Rotation. Electronics, 14.","DOI":"10.3390\/electronics14051036"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Espiga-Fern\u00e1ndez, F., Garc\u00eda-S\u00e1nchez, \u00c1., and Ordieres-Mer\u00e9, J. (2024). A Systematic Approach to Portfolio Optimization: A Comparative Study of Reinforcement Learning Agents, Market Signals, and Investment Horizons. Algorithms, 17.","DOI":"10.3390\/a17120570"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Shabe, R., Engelbrecht, A., and Anderson, K. (2025). Incremental Reinforcement Learning for Portfolio Optimisation. Computers, 14.","DOI":"10.3390\/computers14070242"},{"key":"ref_14","first-page":"10","article-title":"Minimum-Variance Portfolios in the U.S","volume":"33","author":"Clarke","year":"2006","journal-title":"Equity Market. J. Portf. Manag."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"60","DOI":"10.3905\/jpm.2010.36.4.060","article-title":"On the Properties of Equally-Weighted Risk Contributions Portfolios","volume":"36","author":"Maillard","year":"2010","journal-title":"J. Portf. Manag."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1007\/s10957-011-9968-2","article-title":"Entropic Value-at-Risk: A New Coherent Risk Measure","volume":"155","year":"2012","journal-title":"J. Optim. Theory Appl."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1505","DOI":"10.1016\/S0378-4266(02)00281-9","article-title":"Spectral measures of risk: A coherent representation of subjective risk aversion","volume":"26","author":"Acerbi","year":"2002","journal-title":"J. Bank. Financ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1142\/S0219024905002767","article-title":"Drawdown measure in portfolio optimization","volume":"8","author":"Chekhlov","year":"2005","journal-title":"Int. J. Theor. Appl. Financ."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"5176","DOI":"10.3934\/era.2024239","article-title":"Reinforcement Learning for Deep Portfolio Optimization","volume":"32","author":"Yan","year":"2024","journal-title":"Electron. Res. Arch."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, X., and Liu, L. (2025). Risk-Sensitive Deep Reinforcement Learning for Portfolio Optimization. J. Risk Financ. Manag., 18.","DOI":"10.3390\/jrfm18070347"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, X., Liu, S., and Pan, J. (2025). Two-Stage Distributionally Robust Optimization for an Asymmetric Loss-Aversion Portfolio via Deep Learning. Symmetry, 17.","DOI":"10.3390\/sym17081236"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1007\/s11063-024-11582-4","article-title":"Deep Reinforcement Learning Model for Stock Portfolio Management Based on Data Fusion","volume":"56","author":"Li","year":"2024","journal-title":"Neural Process. Lett."},{"key":"ref_23","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press. [2nd ed.]."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1007\/s40747-025-01884-y","article-title":"A novel multi-agent dynamic portfolio optimization learning system based on hierarchical deep reinforcement learning","volume":"11","author":"Sun","year":"2025","journal-title":"Complex Intell. Syst."},{"key":"ref_25","first-page":"1257","article-title":"Hierarchical Multi-Agent System with Bayesian Neural Networks for Portfolio Optimization","volume":"12","author":"Khemlichi","year":"2025","journal-title":"Math. Model. Eng. Probl."},{"key":"ref_26","unstructured":"Bai, Y., Gao, Y., Wan, R., Zhang, S., and Song, R. (2024). A Review of Reinforcement Learning in Financial Applications. arXiv."},{"key":"ref_27","unstructured":"Araci, D. (2019). FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, Z., and Qi, Z. (2023, January 4\u20136). Future Stock Price Prediction Based on Bayesian LSTM in CRSP. Proceedings of the 3rd International Conference on Internet Finance and Digital Economy (ICIFDE 2023), Chengdu, China.","DOI":"10.2991\/978-94-6463-270-5_23"},{"key":"ref_29","unstructured":"Korangi, K., Mues, C., and Bravo, C. (2025). Large-Scale Time-Varying Portfolio Optimisation Using Graph Attention Networks. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Du, S., and Shen, H. (2024). Reinforcement Learning-Based Multimodal Model for the Stock Investment Portfolio Management Task. Electronics, 13.","DOI":"10.3390\/electronics13193895"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1007\/s44196-025-00875-8","article-title":"Risk-Adjusted Deep Reinforcement Learning for Portfolio Optimization: A Multi-reward Approach","volume":"18","author":"Choudhary","year":"2025","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3733714","article-title":"The Evolution of Reinforcement Learning in Quantitative Finance: A Survey","volume":"57","author":"Pippas","year":"2025","journal-title":"ACM Comput. Surv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Khemlichi, F., Elfilali, H.E., Chougrad, H., Ali, S.E.B., and Khamlichi, Y.I. (2023, January 19\u201321). Actor-Critic Methods in Stock Trading: A Comparative Study. Proceedings of the 2023 3rd International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), Tenerife, Spain.","DOI":"10.1109\/ICECCME57830.2023.10253277"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"49","DOI":"10.3905\/jpm.1994.409501","article-title":"The Sharpe Ratio","volume":"21","author":"Sharpe","year":"1994","journal-title":"J. Portf. Manag."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1109\/TNNLS.2016.2522401","article-title":"Deep Direct Reinforcement Learning for Financial Signal Representation and Trading","volume":"28","author":"Deng","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_36","unstructured":"Zhang, Z., Zohren, S., and Roberts, S. (2019). Deep Reinforcement Learning for Trading. arXiv."},{"key":"ref_37","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_38","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2019). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_39","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv."},{"key":"ref_40","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention Is All You Need. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1139","DOI":"10.1111\/j.1540-6261.2007.01232.x","article-title":"Giving Content to Investor Sentiment: The Role of Media in the Stock Market","volume":"62","author":"Tetlock","year":"2007","journal-title":"J. Finance"},{"key":"ref_42","unstructured":"Neal, R.M. (2012). Bayesian Learning for Neural Networks. Lecture Notes in Statistics, Springer."},{"key":"ref_43","unstructured":"Veli\u010dkovi\u0107, P., Cucurull, G., Casanova, A., Romero, A., Li\u00f2, P., and Bengio, Y. (2018). Graph Attention Networks. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Battiston, S., Puliga, M., Kaushik, R., Tasca, P., and Caldarelli, G. (2012). DebtRank: Too Central to Fail? Financial Networks, the FED and Systemic Risk. Sci. Rep., 2.","DOI":"10.1038\/srep00541"},{"key":"ref_45","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (2016). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"21","DOI":"10.21314\/JOR.2000.038","article-title":"Optimization of conditional value-at-risk","volume":"2","author":"Rockafellar","year":"2000","journal-title":"J. Risk"},{"key":"ref_47","unstructured":"Yahoo Finance (2025, August 14). Yahoo Finance\u2014Stock Market Live, Quotes, Business & Finance News. Available online: https:\/\/finance.yahoo.com\/."},{"key":"ref_48","unstructured":"(2025, August 14). Investing.com. Stock Market Quotes & Financial News. Available online: https:\/\/www.investing.com\/."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Lopez de Prado, M. (2018). Advances in Financial Machine Learning, John Wiley & Sons.","DOI":"10.2139\/ssrn.3257415"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Little, R.J.A., and Rubin, D.B. (2019). Statistical Analysis with Missing Data, John Wiley & Sons.","DOI":"10.1002\/9781119482260"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1093\/rapstu\/raaa008","article-title":"The Unprecedented Stock Market Reaction to COVID-19. Rev","volume":"10","author":"Baker","year":"2020","journal-title":"Asset Pricing Stud."},{"key":"ref_52","first-page":"77","article-title":"Portfolio Selection","volume":"7","author":"Markowitz","year":"1952","journal-title":"J. Finance"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1239\/jap\/1077134674","article-title":"On the Maximum Drawdown of a Brownian Motion","volume":"41","author":"Atiya","year":"2004","journal-title":"J. Appl. Probab."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Tsay, R.S. (2010). Analysis of Financial Time Series, John Wiley & Sons. [3rd ed.].","DOI":"10.1002\/9780470644560"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/11\/961\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T17:23:05Z","timestamp":1762363385000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/11\/961"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,5]]},"references-count":54,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["info16110961"],"URL":"https:\/\/doi.org\/10.3390\/info16110961","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,5]]}}}