{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T16:48:57Z","timestamp":1770742137929,"version":"3.49.0"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2021,3,26]],"date-time":"2021-03-26T00:00:00Z","timestamp":1616716800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,26]],"date-time":"2021-03-26T00:00:00Z","timestamp":1616716800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology, Taiwan","doi-asserted-by":"publisher","award":["109-2218-E-001-004"],"award-info":[{"award-number":["109-2218-E-001-004"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016999","name":"Western Norway University Of Applied Sciences","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100016999","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Portfolio management involves position sizing and resource allocation. Traditional and generic portfolio strategies require forecasting of future stock prices as model inputs, which is not a trivial task since those values are difficult to obtain in the real-world applications. To overcome the above limitations and provide a better solution for portfolio management, we developed a Portfolio Management System (PMS) using reinforcement learning with two neural networks (CNN and RNN). A novel reward function involving Sharpe ratios is also proposed to evaluate the performance of the developed systems. Experimental results indicate that the PMS with the Sharpe ratio reward function exhibits outstanding performance, increasing return by 39.0% and decreasing drawdown by 13.7% on average compared to the reward function of trading return. In addition, the proposed  model is more suitable for the construction of a reinforcement learning portfolio, but has 1.98 times more drawdown risk than the . Among the conducted datasets, the PMS outperforms the benchmark strategies in TW50 and traditional stocks, but is inferior to a benchmark strategy in the financial dataset. The PMS is profitable, effective, and offers lower investment risk among almost all datasets. The novel reward function involving the Sharpe ratio enhances performance, and well supports resource-allocation for empirical stock trading.<\/jats:p>","DOI":"10.1007\/s10489-021-02262-0","type":"journal-article","created":{"date-parts":[[2021,3,26]],"date-time":"2021-03-26T07:02:46Z","timestamp":1616742166000},"page":"8119-8131","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":66,"title":["Portfolio management system in equity market neutral using reinforcement learning"],"prefix":"10.1007","volume":"51","author":[{"given":"Mu-En","family":"Wu","sequence":"first","affiliation":[]},{"given":"Jia-Hao","family":"Syu","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8768-9709","authenticated-orcid":false,"given":"Jerry Chun-Wei","family":"Lin","sequence":"additional","affiliation":[]},{"given":"Jan-Ming","family":"Ho","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,3,26]]},"reference":[{"issue":"1","key":"2262_CR1","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/S1566-0141(00)00017-0","volume":"2","author":"A Gunasekarage","year":"2001","unstructured":"Gunasekarage A, Power DM (2001) The profitability of moving average trading rules in south asian stock markets. Emerg Mark Rev 2(1):17\u201333","journal-title":"Emerg Mark Rev"},{"issue":"1","key":"2262_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/jrfm7010001","volume":"7","author":"TTL Chong","year":"2014","unstructured":"Chong TTL, Ng WK, Liew VKS (2014) Revisiting the performance of macd and rsi sscillators. Journal of Risk and Financial Management 7(1):1\u201312","journal-title":"Journal of Risk and Financial Management"},{"key":"2262_CR3","doi-asserted-by":"publisher","first-page":"32061","DOI":"10.1109\/ACCESS.2019.2899177","volume":"7","author":"YC Tsai","year":"2019","unstructured":"Tsai YC, Wu ME, Syu JH, Lei CL, Wu CS, Ho JM, Wang CJ (2019) Assessing the profitability of timely opening range breakout on index futures markets. IEEE Access 7:32061\u201332071","journal-title":"IEEE Access"},{"key":"2262_CR4","doi-asserted-by":"crossref","unstructured":"Syu JH, Wu ME, Lee SH, Ho JM (2019) Modified orb strategies with threshold adjusting on taiwan futures market. In: IEEE conference on computational intelligence for financial engineering & economics, pp 1\u20137","DOI":"10.1109\/CIFEr.2019.8759112"},{"key":"2262_CR5","unstructured":"Reilly FK, Brown KC (2011) Investment analysis and portfolio management. Cengage Learning"},{"issue":"4","key":"2262_CR6","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1111\/1540-5885.1640333","volume":"16","author":"RG Cooper","year":"1999","unstructured":"Cooper RG, Edgett SJ, Kleinschmidt EJ (1999) New product portfolio management: practices and performance. Journal of Product Innovation Managemen 16(4):333\u2013351","journal-title":"Journal of Product Innovation Managemen"},{"issue":"1","key":"2262_CR7","first-page":"77","volume":"7","author":"M Harry","year":"1952","unstructured":"Harry M (1952) Portfolio selection. The Journal of Finance 7(1):77\u201391","journal-title":"The Journal of Finance"},{"issue":"4","key":"2262_CR8","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1002\/j.1538-7305.1956.tb03809.x","volume":"35","author":"JL Kelly","year":"1956","unstructured":"Kelly JL (1956) A new interpretation of information rate. The Bell System Technical Journal 35 (4):917\u2013926","journal-title":"The Bell System Technical Journal"},{"issue":"1","key":"2262_CR9","doi-asserted-by":"publisher","first-page":"31","DOI":"10.2469\/faj.v45.n1.31","volume":"45","author":"RO Michaud","year":"1989","unstructured":"Michaud RO (1989) The markowitz optimization enigma: is optimized optimal?. Financial Analysts Journal 45(1):31\u201342","journal-title":"Financial Analysts Journal"},{"issue":"7","key":"2262_CR10","doi-asserted-by":"publisher","first-page":"2495","DOI":"10.1093\/rfs\/hhn113","volume":"22","author":"AJ Patton","year":"2009","unstructured":"Patton AJ (2009) Are market neutral hedge funds really market neutral?. The Review of Financial Studies 22(7):2495\u2013 2530","journal-title":"The Review of Financial Studies"},{"key":"2262_CR11","volume-title":"Reinforcement learning and dynamic programming using function approximators, vol 39","author":"L Busoniu","year":"2010","unstructured":"Busoniu L, Babuska R, De Schutter B, Ernst D (2010) Reinforcement learning and dynamic programming using function approximators, vol 39. CRC Press, Boca Raton"},{"key":"2262_CR12","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, Cambridge"},{"key":"2262_CR13","volume-title":"Neural networks: a comprehensive foundation","author":"S Haykin","year":"2007","unstructured":"Haykin S (2007) Neural networks: a comprehensive foundation. Prentice-Hall, Inc., Englewood Cliffs"},{"key":"2262_CR14","volume-title":"Neural networks for financial forecasting","author":"E Gately","year":"1995","unstructured":"Gately E (1995) Neural networks for financial forecasting. Wiley, New York"},{"issue":"7553","key":"2262_CR15","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436\u2013444","journal-title":"Nature"},{"key":"2262_CR16","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","volume":"61","author":"J Schmidhuber","year":"2015","unstructured":"Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85\u2013117","journal-title":"Neural Netw"},{"key":"2262_CR17","unstructured":"Zaremba W, Sutskever I, Vinyals O (2014) Recurrent neural network regularization. arXiv:1409.2329"},{"key":"2262_CR18","doi-asserted-by":"publisher","first-page":"106548","DOI":"10.1016\/j.knosys.2020.106548","volume":"212","author":"JCW Lin","year":"2021","unstructured":"Lin JCW, Shao Y, Djenouri Y, Yun U (2021) ASRNN: a recurrent neural network with an attention model for sequence labeling. Knowl-Based Syst 212:106548","journal-title":"Knowl-Based Syst"},{"key":"2262_CR19","doi-asserted-by":"crossref","unstructured":"Qin Z, Yu F, Liu C, Chen X (2018) How convolutional neural network see the world-a survey of convolutional neural network visualization methods. arXiv:1804.11191","DOI":"10.3934\/mfc.2018008"},{"key":"2262_CR20","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.neucom.2016.12.038","volume":"234","author":"W Liu","year":"2017","unstructured":"Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11\u201326","journal-title":"Neurocomputing"},{"key":"2262_CR21","doi-asserted-by":"crossref","unstructured":"Jiang Z, Liang J (2017) Cryptocurrency portfolio management with deep reinforcement learning. In: Intelligent systems conference, pp 905\u2013913","DOI":"10.1109\/IntelliSys.2017.8324237"},{"key":"2262_CR22","unstructured":"Jiang Z, Xu D, Liang J (2017) A deep reinforcement learning framework for the financial portfolio management problem. arXiv:1706.10059"},{"key":"2262_CR23","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1016\/j.eswa.2017.06.023","volume":"87","author":"S Almahdi","year":"2017","unstructured":"Almahdi S, Yang SY (2017) An adaptive portfolio trading system: a risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown. Expert Syst Appl 87:267\u2013279","journal-title":"Expert Syst Appl"},{"issue":"1","key":"2262_CR24","doi-asserted-by":"publisher","first-page":"49","DOI":"10.3905\/jpm.1994.409501","volume":"21","author":"WF Sharpe","year":"1994","unstructured":"Sharpe WF (1994) The sharpe ratio. J Portf Manag 21(1):49\u201358","journal-title":"J Portf Manag"},{"issue":"2","key":"2262_CR25","doi-asserted-by":"publisher","first-page":"3","DOI":"10.21314\/JOR.2012.255","volume":"15","author":"DH Bailey","year":"2012","unstructured":"Bailey DH, Lopez de Prado M (2012) The sharpe ratio efficient frontier. Journal of Risk 15 (2):3\u201344","journal-title":"Journal of Risk"},{"issue":"10","key":"2262_CR26","first-page":"99","volume":"17","author":"M Magdon Ismail","year":"2004","unstructured":"Magdon Ismail M, Atiya AF (2004) Maximum drawdown. Risk Magazine 17(10):99\u2013102","journal-title":"Risk Magazine"},{"key":"2262_CR27","doi-asserted-by":"crossref","unstructured":"Kozat SS, Singer AC (2007) Universal constant rebalanced portfolios with switching. In: IEEE international conference on acoustics, speech and signal processing, vol 3, pp 1129\u20131132","DOI":"10.1109\/ICASSP.2007.366883"},{"issue":"5","key":"2262_CR28","doi-asserted-by":"publisher","first-page":"2129","DOI":"10.1111\/j.1540-6261.1997.tb02755.x","volume":"52","author":"AJ Richards","year":"1997","unstructured":"Richards AJ (1997) Winner-loser reversals in national stock market indices: can they be explained?. The Journal of Finance 52(5):2129\u20132144","journal-title":"The Journal of Finance"},{"issue":"9","key":"2262_CR29","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1080\/09603100600722193","volume":"17","author":"A Siganos","year":"2007","unstructured":"Siganos A (2007) Momentum returns and size of winner and loser portfolios. Appl Financ Econ 17(9):701\u2013708","journal-title":"Appl Financ Econ"},{"key":"2262_CR30","doi-asserted-by":"crossref","unstructured":"Alexander C, Dimitriu A (2002) The cointegration alpha: enhanced index tracking and long-short equity market neutral strategies. ISMA Finance Discussion Paper","DOI":"10.2139\/ssrn.315619"},{"key":"2262_CR31","unstructured":"Jeng Y, Ton W, Lee KJ, Chuang HM (2006) Taiwan multi-factor model construction: equity market neutral strategies application. Managerial Finance"},{"key":"2262_CR32","unstructured":"Pai GV, Michel T (2012) Differential evolution based optimization of risk budgeted equity market neutral portfolios. In: IEEE congress on evolutionary computation, pp 1\u20138"},{"key":"2262_CR33","doi-asserted-by":"crossref","unstructured":"Van Otterlo M, Wiering M (2012) Reinforcement learning and markov decision processes. In: Reinforcement learning. Springer, pp 3\u201342","DOI":"10.1007\/978-3-642-27645-3_1"},{"key":"2262_CR34","doi-asserted-by":"crossref","unstructured":"Malkiel BG (1989) Efficient market hypothesis. In: Finance, society for financial studies, pp 127\u2013134","DOI":"10.1007\/978-1-349-20213-3_13"},{"key":"2262_CR35","first-page":"1057","volume":"12","author":"RS Sutton","year":"1999","unstructured":"Sutton RS, McAllester D, Singh S, Mansour Y (1999) Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems 12:1057\u20131063","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2262_CR36","unstructured":"Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M (2014) Deterministic policy gradient algorithms. In: International conference on machine learning"},{"issue":"1","key":"2262_CR37","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.datak.2010.09.002","volume":"70","author":"Z Huang","year":"2011","unstructured":"Huang Z, van der Aalst WM, Lu X, Duan H (2011) Reinforcement learning based resource allocation in business process management. Data & Knowledge Engineering 70(1):127\u2013 145","journal-title":"Data & Knowledge Engineering"},{"key":"2262_CR38","unstructured":"Smart WD, Kaelbling LP (2002) Effective reinforcement learning for mobile robots. In: IEEE international conference on robotics and automation, vol 4, pp 3404\u20133410"},{"issue":"4","key":"2262_CR39","doi-asserted-by":"publisher","first-page":"875","DOI":"10.1109\/72.935097","volume":"12","author":"J Moody","year":"2001","unstructured":"Moody J, Saffell M (2001) Learning to trade via direct reinforcement. IEEE Transactions on Neural Networks 12(4):875\u2013 889","journal-title":"IEEE Transactions on Neural Networks"},{"key":"2262_CR40","unstructured":"Lee JW (2001) Stock price prediction using reinforcement learning. In: IEEE international symposium on industrial electronics proceedings, vol 1, pp 690\u2013695"},{"key":"2262_CR41","doi-asserted-by":"crossref","unstructured":"Tsantekidis A, Passalis N, Tefas A, Kanniainen J, Gabbouj M, Iosifidis A (2017) Forecasting stock prices from the limit order book using convolutional neural networks. In: IEEE conference on business informatics, vol 1, pp 7\u201312","DOI":"10.1109\/CBI.2017.23"},{"key":"2262_CR42","doi-asserted-by":"crossref","unstructured":"Chen JF, Chen WL, Huang CP, Huang SH, Chen AP (2016) Financial time-series data analysis using deep convolutional neural networks. In: International conference on cloud computing and big data, pp 87\u201392","DOI":"10.1109\/CCBD.2016.027"},{"key":"2262_CR43","doi-asserted-by":"crossref","unstructured":"Mesnil G, He X, Deng L, Bengio Y (2013) Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding. In: Interspeech, pp 3771\u2013 3775","DOI":"10.21437\/Interspeech.2013-596"},{"issue":"10","key":"2262_CR44","doi-asserted-by":"publisher","first-page":"2833","DOI":"10.1162\/neco_a_01124","volume":"30","author":"T Gao","year":"2018","unstructured":"Gao T, Chai Y (2018) Improving stock closing price prediction using recurrent neural network and technical indicators. Neural Comput 30(10):2833\u20132854","journal-title":"Neural Comput"},{"key":"2262_CR45","doi-asserted-by":"crossref","unstructured":"Wang J, Wang J, Fang W, Niu H (2016) Financial time series prediction using elman recurrent random neural networks. Computational Intelligence and Neuroscience 2016","DOI":"10.1155\/2016\/4742515"},{"issue":"4","key":"2262_CR46","doi-asserted-by":"publisher","first-page":"536","DOI":"10.1016\/j.jksuci.2015.06.002","volume":"29","author":"AK Rout","year":"2017","unstructured":"Rout AK, Dash PK, Dash R, Bisoi R (2017) Forecasting financial time series using a low complexity recurrent neural network and evolutionary learning approach. Journal of King Saud University-Computer and Information Sciences 29(4):536\u2013552","journal-title":"Journal of King Saud University-Computer and Information Sciences"},{"issue":"18","key":"2262_CR47","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1080\/09603100500389630","volume":"15","author":"CC Lin","year":"2005","unstructured":"Lin CC, Chiang MH (2005) Volatility effect of etfs on the constituents of the underlying Taiwan 50 index. Appl Financ Econ 15(18):1315\u20131322","journal-title":"Appl Financ Econ"},{"issue":"2","key":"2262_CR48","doi-asserted-by":"publisher","first-page":"755","DOI":"10.1111\/jofi.12601","volume":"73","author":"JE Engelberg","year":"2018","unstructured":"Engelberg JE, Reed AV, Ringgenberg MC (2018) Short-selling risk. The Journal of Finance 73(2):755\u2013786","journal-title":"The Journal of Finance"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-021-02262-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-021-02262-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-021-02262-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,7]],"date-time":"2021-10-07T06:02:14Z","timestamp":1633586534000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-021-02262-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,26]]},"references-count":48,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["2262"],"URL":"https:\/\/doi.org\/10.1007\/s10489-021-02262-0","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"value":"0924-669X","type":"print"},{"value":"1573-7497","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,26]]},"assertion":[{"value":"4 February 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 March 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}