{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T09:30:18Z","timestamp":1777714218000,"version":"3.51.4"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,8,28]],"date-time":"2020-08-28T00:00:00Z","timestamp":1598572800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,8,28]],"date-time":"2020-08-28T00:00:00Z","timestamp":1598572800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100008095","name":"Carleton University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008095","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In the era of big data, deep learning for predicting stock market prices and trends has become even more popular than before. We collected 2\u00a0years of data from Chinese stock market and proposed a comprehensive customization of feature engineering and deep learning-based model for predicting price trend of stock markets. The proposed solution is comprehensive as it includes pre-processing of the stock market dataset, utilization of multiple feature engineering techniques, combined with a customized deep learning based system for stock market price trend prediction. We conducted comprehensive evaluations on frequently used machine learning models and conclude that our proposed solution outperforms due to the comprehensive feature engineering that we built. The system achieves overall high accuracy for stock market trend prediction. With the detailed design and evaluation of prediction term lengths, feature engineering, and data pre-processing methods, this work contributes to the stock analysis research community both in the financial and technical domains.<\/jats:p>","DOI":"10.1186\/s40537-020-00333-6","type":"journal-article","created":{"date-parts":[[2020,8,28]],"date-time":"2020-08-28T12:03:05Z","timestamp":1598616185000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":303,"title":["Short-term stock market price trend prediction using a comprehensive deep learning system"],"prefix":"10.1186","volume":"7","author":[{"given":"Jingyi","family":"Shen","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1859-8296","authenticated-orcid":false,"given":"M. Omair","family":"Shafiq","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,8,28]]},"reference":[{"issue":"7","key":"333_CR1","doi-asserted-by":"publisher","first-page":"10696","DOI":"10.1016\/j.eswa.2009.02.043","volume":"36","author":"GS Atsalakis","year":"2009","unstructured":"Atsalakis GS, Valavanis KP. Forecasting stock market short-term trends using a neuro-fuzzy based methodology. Expert Syst Appl. 2009;36(7):10696\u2013707.","journal-title":"Expert Syst Appl"},{"key":"333_CR2","doi-asserted-by":"publisher","unstructured":"Ayo CK. Stock price prediction using the ARIMA model. In: 2014 UKSim-AMSS 16th international conference on computer modelling and simulation. 2014. https:\/\/doi.org\/10.1109\/UKSim.2014.67.","DOI":"10.1109\/UKSim.2014.67"},{"key":"333_CR3","unstructured":"Brownlee J. Deep learning for time series forecasting: predict the future with MLPs, CNNs and LSTMs in Python. Machine Learning Mastery. 2018. https:\/\/machinelearningmastery.com\/time-series-prediction-lstm-recurrent-neural-networks-python-keras\/"},{"key":"333_CR4","doi-asserted-by":"publisher","unstructured":"Eapen J, Bein D, Verma A. Novel deep learning model with CNN and bi-directional LSTM for improved stock market index prediction. In: 2019 IEEE 9th annual computing and communication workshop and conference (CCWC). 2019. pp. 264\u201370. https:\/\/doi.org\/10.1109\/CCWC.2019.8666592.","DOI":"10.1109\/CCWC.2019.8666592"},{"issue":"2","key":"333_CR5","doi-asserted-by":"publisher","first-page":"654","DOI":"10.1016\/j.ejor.2017.11.054","volume":"270","author":"T Fischer","year":"2018","unstructured":"Fischer T, Krauss C. Deep learning with long short-term memory networks for financial market predictions. Eur J Oper Res. 2018;270(2):654\u201369. https:\/\/doi.org\/10.1016\/j.ejor.2017.11.054.","journal-title":"Eur J Oper Res"},{"key":"333_CR6","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1023\/A:1012487302797","volume":"46","author":"I Guyon","year":"2002","unstructured":"Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Mach Learn 2002;46:389\u2013422.","journal-title":"Mach Learn"},{"key":"333_CR7","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1016\/j.asoc.2014.12.028","volume":"29","author":"R Hafezi","year":"2015","unstructured":"Hafezi R, Shahrabi J, Hadavandi E. A bat-neural network multi-agent system (BNNMAS) for stock price prediction: case study of DAX stock price. Appl Soft Comput J. 2015;29:196\u2013210. https:\/\/doi.org\/10.1016\/j.asoc.2014.12.028.","journal-title":"Appl Soft Comput J"},{"issue":"2","key":"333_CR8","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1137\/090771806","volume":"53","author":"N Halko","year":"2001","unstructured":"Halko N, Martinsson PG, Tropp JA. Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 2001;53(2):217\u201388.","journal-title":"SIAM Rev"},{"key":"333_CR9","doi-asserted-by":"publisher","unstructured":"Hassan MR, Nath B. Stock market forecasting using Hidden Markov Model: a new approach. In: Proceedings\u20145th international conference on intelligent systems design and applications 2005, ISDA\u201905. 2005. pp. 192\u20136. https:\/\/doi.org\/10.1109\/ISDA.2005.85.","DOI":"10.1109\/ISDA.2005.85"},{"issue":"8","key":"333_CR10","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J. Long short-term memory. J Neural Comput. 1997;9(8):1735\u201380. https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735.","journal-title":"J Neural Comput"},{"issue":"3\u20134","key":"333_CR11","doi-asserted-by":"publisher","first-page":"651","DOI":"10.1007\/s00521-011-0721-4","volume":"22","author":"CM Hsu","year":"2013","unstructured":"Hsu CM. A hybrid procedure with feature selection for resolving stock\/futures price forecasting problems. Neural Comput Appl. 2013;22(3\u20134):651\u201371. https:\/\/doi.org\/10.1007\/s00521-011-0721-4.","journal-title":"Neural Comput Appl"},{"issue":"1","key":"333_CR12","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/J.POLYMER.2016.08.021","volume":"14","author":"CF Huang","year":"2012","unstructured":"Huang CF, Chang BR, Cheng DW, Chang CH. Feature selection and parameter optimization of a fuzzy-based stock selection model using genetic algorithms. Int J Fuzzy Syst. 2012;14(1):65\u201375. https:\/\/doi.org\/10.1016\/J.POLYMER.2016.08.021.","journal-title":"Int J Fuzzy Syst"},{"issue":"2 PART 1","key":"333_CR13","doi-asserted-by":"publisher","first-page":"1529","DOI":"10.1016\/j.eswa.2007.11.062","volume":"36","author":"CL Huang","year":"2009","unstructured":"Huang CL, Tsai CY. A hybrid SOFM-SVR with a filter-based feature selection for stock market forecasting. Expert Syst Appl. 2009;36(2 PART 1):1529\u201339. https:\/\/doi.org\/10.1016\/j.eswa.2007.11.062.","journal-title":"Expert Syst Appl"},{"key":"333_CR14","doi-asserted-by":"publisher","first-page":"17287","DOI":"10.1109\/ACCESS.2019.2895252","volume":"7","author":"SM Idrees","year":"2019","unstructured":"Idrees SM, Alam MA, Agarwal P. A prediction approach for stock market volatility based on time series data. IEEE Access. 2019;7:17287\u201398. https:\/\/doi.org\/10.1109\/ACCESS.2019.2895252.","journal-title":"IEEE Access"},{"key":"333_CR15","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1080\/03081070601068595","volume":"37","author":"H Ince","year":"2008","unstructured":"Ince H, Trafalis TB. Short term forecasting with support vector machines and application to stock price prediction. Int J Gen Syst. 2008;37:677\u201387. https:\/\/doi.org\/10.1080\/03081070601068595.","journal-title":"Int J Gen Syst"},{"key":"333_CR16","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/j.future.2017.02.010","volume":"80","author":"S Jeon","year":"2018","unstructured":"Jeon S, Hong B, Chang V. Pattern graph tracking-based stock price prediction using big data. Future Gener Comput Syst. 2018;80:171\u201387. https:\/\/doi.org\/10.1016\/j.future.2017.02.010.","journal-title":"Future Gener Comput Syst"},{"issue":"5","key":"333_CR17","doi-asserted-by":"publisher","first-page":"5311","DOI":"10.1016\/j.eswa.2010.10.027","volume":"38","author":"Y Kara","year":"2011","unstructured":"Kara Y, Acar Boyacioglu M, Baykan \u00d6K. Predicting direction of stock price index movement using artificial neural networks and support vector machines: the sample of the Istanbul Stock Exchange. Expert Syst Appl. 2011;38(5):5311\u20139. https:\/\/doi.org\/10.1016\/j.eswa.2010.10.027.","journal-title":"Expert Syst Appl"},{"key":"333_CR18","unstructured":"Khaidem L, Dey SR. Predicting the direction of stock market prices using random forest. 2016. pp. 1\u201320."},{"key":"333_CR19","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1016\/S0957-4174(00)00027-0","volume":"19","author":"K Kim","year":"2000","unstructured":"Kim K, Han I. Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index. Expert Syst Appl. 2000;19:125\u201332. https:\/\/doi.org\/10.1016\/S0957-4174(00)00027-0.","journal-title":"Expert Syst Appl"},{"issue":"8","key":"333_CR20","doi-asserted-by":"publisher","first-page":"10896","DOI":"10.1016\/j.eswa.2009.02.038","volume":"36","author":"MC Lee","year":"2009","unstructured":"Lee MC. Using support vector machine with a hybrid feature selection method to the stock trend prediction. Expert Syst Appl. 2009;36(8):10896\u2013904. https:\/\/doi.org\/10.1016\/j.eswa.2009.02.038.","journal-title":"Expert Syst Appl"},{"key":"333_CR21","doi-asserted-by":"publisher","first-page":"923","DOI":"10.1016\/j.asoc.2017.09.029","volume":"62","author":"L Lei","year":"2018","unstructured":"Lei L. Wavelet neural network prediction method of stock price trend based on rough set attribute reduction. Appl Soft Comput J. 2018;62:923\u201332. https:\/\/doi.org\/10.1016\/j.asoc.2017.09.029.","journal-title":"Appl Soft Comput J"},{"issue":"3","key":"333_CR22","doi-asserted-by":"publisher","first-page":"7313","DOI":"10.1016\/j.eswa.2008.09.049","volume":"36","author":"X Lin","year":"2009","unstructured":"Lin X, Yang Z, Song Y. Expert systems with applications short-term stock price prediction based on echo state networks. Expert Syst Appl. 2009;36(3):7313\u20137. https:\/\/doi.org\/10.1016\/j.eswa.2008.09.049.","journal-title":"Expert Syst Appl"},{"issue":"March","key":"333_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.engappai.2019.03.019","volume":"82","author":"G Liu","year":"2019","unstructured":"Liu G, Wang X. A new metric for individual stock trend prediction. Eng Appl Artif Intell. 2019;82(March):1\u201312. https:\/\/doi.org\/10.1016\/j.engappai.2019.03.019.","journal-title":"Eng Appl Artif Intell"},{"key":"333_CR24","doi-asserted-by":"publisher","unstructured":"Liu S, Zhang C, Ma J. CNN-LSTM neural network model for quantitative strategy analysis in stock markets. 2017;1:198\u2013206. https:\/\/doi.org\/10.1007\/978-3-319-70096-0.","DOI":"10.1007\/978-3-319-70096-0"},{"key":"333_CR25","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1016\/j.knosys.2018.10.034","volume":"164","author":"W Long","year":"2018","unstructured":"Long W, Lu Z, Cui L. Deep learning-based feature engineering for stock price movement prediction. Knowl Based Syst. 2018;164:163\u201373. https:\/\/doi.org\/10.1016\/j.knosys.2018.10.034.","journal-title":"Knowl Based Syst"},{"issue":"2","key":"333_CR26","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1111\/j.1540-6261.1970.tb00518.x","volume":"25","author":"BG Malkiel","year":"1970","unstructured":"Malkiel BG, Fama EF. Efficient capital markets: a review of theory and empirical work. J Finance. 1970;25(2):383\u2013417.","journal-title":"J Finance"},{"key":"333_CR27","doi-asserted-by":"publisher","unstructured":"McNally S, Roche J, Caton S. Predicting the price of bitcoin using machine learning. In: Proceedings\u201426th Euromicro international conference on parallel, distributed, and network-based processing, PDP 2018. pp. 339\u201343. https:\/\/doi.org\/10.1109\/PDP2018.2018.00060.","DOI":"10.1109\/PDP2018.2018.00060"},{"key":"333_CR28","unstructured":"Nagar A, Hahsler M. News sentiment analysis using R to predict stock market trends. 2012. http:\/\/past.rinfinance.com\/agenda\/2012\/talk\/Nagar+Hahsler.pdf. Accessed 20 July 2019."},{"key":"333_CR29","doi-asserted-by":"crossref","unstructured":"Nekoeiqachkanloo H, Ghojogh B, Pasand AS, Crowley M. Artificial counselor system for stock investment. 2019. ArXiv Preprint arXiv:1903.00955.","DOI":"10.1609\/aaai.v33i01.33019558"},{"issue":"5","key":"333_CR30","doi-asserted-by":"publisher","first-page":"5569","DOI":"10.1016\/j.eswa.2010.10.079","volume":"38","author":"LP Ni","year":"2011","unstructured":"Ni LP, Ni ZW, Gao YZ. Stock trend prediction based on fractal feature selection and support vector machine. Expert Syst Appl. 2011;38(5):5569\u201376. https:\/\/doi.org\/10.1016\/j.eswa.2010.10.079.","journal-title":"Expert Syst Appl"},{"key":"333_CR31","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-017-2228-y","author":"X Pang","year":"2018","unstructured":"Pang X, Zhou Y, Wang P, Lin W, Chang V. An innovative neural network approach for stock market prediction. J Supercomput. 2018. https:\/\/doi.org\/10.1007\/s11227-017-2228-y.","journal-title":"J Supercomput"},{"issue":"1","key":"333_CR32","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/s10614-017-9665-9","volume":"52","author":"A Pimenta","year":"2018","unstructured":"Pimenta A, Nametala CAL, Guimar\u00e3es FG, Carrano EG. An automated investing method for stock market based on multiobjective genetic programming. Comput Econ. 2018;52(1):125\u201344. https:\/\/doi.org\/10.1007\/s10614-017-9665-9.","journal-title":"Comput Econ"},{"issue":"2","key":"333_CR33","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1016\/S0377-2217(02)00911-6","volume":"156","author":"S Piramuthu","year":"2004","unstructured":"Piramuthu S. Evaluating feature selection methods for learning in data mining applications. Eur J Oper Res. 2004;156(2):483\u201394. https:\/\/doi.org\/10.1016\/S0377-2217(02)00911-6.","journal-title":"Eur J Oper Res"},{"issue":"5","key":"333_CR34","doi-asserted-by":"publisher","first-page":"e0155133","DOI":"10.1371\/journal.pone.0155133","volume":"11","author":"M Qiu","year":"2016","unstructured":"Qiu M, Song Y. Predicting the direction of stock market index movement using an optimized artificial neural network model. PLoS ONE. 2016;11(5):e0155133.","journal-title":"PLoS ONE"},{"key":"333_CR35","unstructured":"Scikit-learn. Scikit-learn Min-Max Scaler. 2019. https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.MinMaxScaler.html. Retrieved 26 July 2020."},{"key":"333_CR36","doi-asserted-by":"crossref","unstructured":"Shen J. Thesis, \u201cShort-term stock market price trend prediction using a customized deep learning system\u201d, supervised by M. Omair Shafiq, Carleton University. 2019.","DOI":"10.1186\/s40537-020-00333-6"},{"key":"333_CR37","first-page":"572","volume":"2018","author":"J Shen","year":"2018","unstructured":"Shen J, Shafiq MO. Deep learning convolutional neural networks with dropout\u2014a parallel approach. ICMLA. 2018;2018:572\u20137.","journal-title":"ICMLA"},{"key":"333_CR38","first-page":"287","volume":"2019","author":"J Shen","year":"2019","unstructured":"Shen J, Shafiq MO. Learning mobile application usage\u2014a deep learning approach. ICMLA. 2019;2019:287\u201392.","journal-title":"ICMLA"},{"key":"333_CR39","doi-asserted-by":"crossref","unstructured":"Shih D. A study of early warning system in volume burst risk assessment of stock with Big Data platform. In: 2019 IEEE 4th international conference on cloud computing and big data analysis (ICCCBDA). 2019. pp. 244\u20138.","DOI":"10.1109\/ICCCBDA.2019.8725738"},{"key":"333_CR40","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3141294","author":"J Sirignano","year":"2018","unstructured":"Sirignano J, Cont R. Universal features of price formation in financial markets: perspectives from deep learning. Ssrn. 2018. https:\/\/doi.org\/10.2139\/ssrn.3141294.","journal-title":"Ssrn"},{"key":"333_CR41","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1016\/j.asoc.2018.03.006","volume":"67","author":"M Thakur","year":"2018","unstructured":"Thakur M, Kumar D. A hybrid financial trading support system using multi-category classifiers and random forest. Appl Soft Comput J. 2018;67:337\u201349. https:\/\/doi.org\/10.1016\/j.asoc.2018.03.006.","journal-title":"Appl Soft Comput J"},{"issue":"1","key":"333_CR42","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1016\/j.dss.2010.08.028","volume":"50","author":"CF Tsai","year":"2010","unstructured":"Tsai CF, Hsiao YC. Combining multiple feature selection methods for stock prediction: union, intersection, and multi-intersection approaches. Decis Support Syst. 2010;50(1):258\u201369. https:\/\/doi.org\/10.1016\/j.dss.2010.08.028.","journal-title":"Decis Support Syst"},{"key":"333_CR43","unstructured":"Tushare API. 2018. https:\/\/github.com\/waditu\/tushare. Accessed 1 July 2019."},{"key":"333_CR44","unstructured":"Wang X, Lin W. Stock market prediction using neural networks: does trading volume help in short-term prediction?. n.d."},{"key":"333_CR45","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1016\/j.eswa.2018.06.016","volume":"112","author":"B Weng","year":"2018","unstructured":"Weng B, Lu L, Wang X, Megahed FM, Martinez W. Predicting short-term stock prices using ensemble methods and online data sources. Expert Syst Appl. 2018;112:258\u201373. https:\/\/doi.org\/10.1016\/j.eswa.2018.06.016.","journal-title":"Expert Syst Appl"},{"key":"333_CR46","unstructured":"Zhang S. Architectural complexity measures of recurrent neural networks, (NIPS). 2016. pp. 1\u20139."},{"key":"333_CR47","doi-asserted-by":"publisher","DOI":"10.1007\/s10588-019-09292-7","volume-title":"Development of stock market trend prediction system using multiple regression. Computational and mathematical organization theory","author":"M Zubair","year":"2019","unstructured":"Zubair M, Fazal A, Fazal R, Kundi M. Development of stock market trend prediction system using multiple regression. Computational and mathematical organization theory. Berlin: Springer US; 2019. https:\/\/doi.org\/10.1007\/s10588-019-09292-7."}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-020-00333-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-020-00333-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-020-00333-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,27]],"date-time":"2021-08-27T23:15:00Z","timestamp":1630106100000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-020-00333-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,28]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["333"],"URL":"https:\/\/doi.org\/10.1186\/s40537-020-00333-6","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,28]]},"assertion":[{"value":"24 January 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 July 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 August 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"66"}}