{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,6]],"date-time":"2025-08-06T12:16:17Z","timestamp":1754482577241,"version":"3.37.3"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T00:00:00Z","timestamp":1712534400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T00:00:00Z","timestamp":1712534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Price prediction remains a crucial aspect of financial market research as it forms the basis for various trading strategies and portfolio management techniques. However, traditional models such as ARIMA are not effective for multi-horizon forecasting, and current deep learning approaches do not take into account the conditional heteroscedasticity of financial market time series. In this work, we introduce the similarity embedded temporal Transformer (SeTT) algorithms, which extend the state-of-the-art temporal Transformer architecture. These algorithms utilise historical trends in financial time series, as well as statistical principles, to enhance forecasting performance. We conducted a thorough analysis of various hyperparameters including learning rate, local window size, and the choice of similarity function in this extension of the study in a bid to get optimal model performance. We also experimented over an extended timeframe, which allowed us to more accurately assess the performance of the models in different market conditions and across different lengths of time. Overall, our results show that SeTT provides improved performance for financial market prediction, as it outperforms both classical financial models and state-of-the-art deep learning methods, across volatile and non-volatile extrapolation periods, with varying effects of historical volatility on the extrapolation. Despite the availability of a substantial amount of data spanning up to 13\u00a0years, optimal results were primarily attained through a historical window of 1\u20133\u00a0years for the extrapolation period under examination.<\/jats:p>","DOI":"10.1007\/s40747-024-01400-8","type":"journal-article","created":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T06:02:13Z","timestamp":1712556133000},"page":"4793-4815","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Towards efficient similarity embedded temporal Transformers via extended timeframe analysis"],"prefix":"10.1007","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1815-0555","authenticated-orcid":false,"given":"Kenniy","family":"Olorunnimbe","sequence":"first","affiliation":[]},{"given":"Herna","family":"Viktor","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,4,8]]},"reference":[{"key":"1400_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2020.106181","volume":"90","author":"OB Sezer","year":"2020","unstructured":"Sezer OB, Gudelek MU, Ozbayoglu AM (2020) Financial time series forecasting with deep learning: a systematic literature review: 2005\u20132019. Appl Soft Comput 90:106181. https:\/\/doi.org\/10.1016\/j.asoc.2020.106181","journal-title":"Appl Soft Comput"},{"key":"1400_CR2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-022-10226-0","author":"K Olorunnimbe","year":"2022","unstructured":"Olorunnimbe K, Viktor HL (2022) Deep learning in the stock market\u2014a systematic survey of practice, backtesting and applications. Artif Intell Rev. https:\/\/doi.org\/10.1007\/s10462-022-10226-0","journal-title":"Artif Intell Rev"},{"issue":"1","key":"1400_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2016.18","volume":"3","author":"MD Wilkinson","year":"2016","unstructured":"Wilkinson MD, Dumontier M, Aalbersberg IJ et al (2016) The FAIR guiding principles for scientific data management and stewardship. Sci Data 3(1):1\u20139. https:\/\/doi.org\/10.1038\/sdata.2016.18","journal-title":"Sci Data"},{"key":"1400_CR4","doi-asserted-by":"publisher","unstructured":"Ouyang L, Wu J, Jiang X, Almeida D, Wainwright CL, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A, Schulman J, Hilton J, Kelton F, Miller L, Simens M, Askell A, Welinder P, Christiano P, Leike J, Lowe R (2022) Training language models to follow instructions with human feedback. https:\/\/doi.org\/10.48550\/arXiv.2203.02155","DOI":"10.48550\/arXiv.2203.02155"},{"issue":"4","key":"1400_CR5","doi-asserted-by":"publisher","first-page":"1748","DOI":"10.1016\/j.ijforecast.2021.03.012","volume":"37","author":"B Lim","year":"2021","unstructured":"Lim B, Arik SO, Loeff N, Pfister T (2021) Temporal fusion transformers for interpretable multi-horizon time series forecasting. Int J Forecast 37(4):1748\u20131764. https:\/\/doi.org\/10.1016\/j.ijforecast.2021.03.012","journal-title":"Int J Forecast"},{"key":"1400_CR6","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-022-10148-x","author":"A de Santana Correia","year":"2022","unstructured":"de Santana Correia A, Colombini EL (2022) Attention, please! a survey of neural attention models in deep learning. Artif Intell Rev. https:\/\/doi.org\/10.1007\/s10462-022-10148-x","journal-title":"Artif Intell Rev"},{"issue":"6","key":"1400_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3530811","volume":"55","author":"Y Tay","year":"2023","unstructured":"Tay Y, Dehghani M, Bahri D, Metzler D (2023) Efficient transformers: a survey. ACM Comput Surv 55(6):1\u201328. https:\/\/doi.org\/10.1145\/3530811","journal-title":"ACM Comput Surv"},{"key":"1400_CR8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1706.03762","author":"A Vaswani","year":"2017","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst. https:\/\/doi.org\/10.48550\/arXiv.1706.03762","journal-title":"Adv Neural Inf Process Syst"},{"key":"1400_CR9","doi-asserted-by":"publisher","unstructured":"Li S, Jin X, Xuan Y, Zhou X, Chen W, Wang Y-X, Yan X (2019) Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In: Proceedings of the 33rd international conference on neural information processing systems. Curran Associates Inc., New York, pp 5243\u20135253. https:\/\/doi.org\/10.48550\/arXiv.1907.00235","DOI":"10.48550\/arXiv.1907.00235"},{"key":"1400_CR10","doi-asserted-by":"publisher","unstructured":"Olorunnimbe K, Viktor HL (2022) Similarity embedded temporal transformers: enhancing stock predictions with historically similar trends. In: 26th International symposium on methodologies for intelligent systems (ISMIS), pp 388\u2013398. https:\/\/doi.org\/10.1007\/978-3-031-16564-1_37","DOI":"10.1007\/978-3-031-16564-1_37"},{"key":"1400_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.113456","volume":"156","author":"F Soleymani","year":"2020","unstructured":"Soleymani F, Paquet E (2020) Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder-DeepBreath. Expert Syst Appl 156:113456. https:\/\/doi.org\/10.1016\/j.eswa.2020.113456","journal-title":"Expert Syst Appl"},{"key":"1400_CR12","doi-asserted-by":"publisher","unstructured":"Wen Q, Zhou T, Zhang C, Chen W, Ma Z, Yan J, Sun L (2022) Transformers in time series: a survey. https:\/\/doi.org\/10.48550\/arXiv.2202.07125","DOI":"10.48550\/arXiv.2202.07125"},{"key":"1400_CR13","doi-asserted-by":"publisher","first-page":"131662","DOI":"10.1109\/access.2020.3009626","volume":"8","author":"K Mishev","year":"2020","unstructured":"Mishev K, Gjorgjevikj A, Vodenska I, Chitkushev LT, Trajanov D (2020) Evaluation of sentiment analysis in finance: from lexicons to transformers. IEEE Access 8:131662\u2013131682. https:\/\/doi.org\/10.1109\/access.2020.3009626","journal-title":"IEEE Access"},{"key":"1400_CR14","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118128","volume":"208","author":"C Wang","year":"2022","unstructured":"Wang C, Chen Y, Zhang S, Zhang Q (2022) Stock market index prediction using deep transformer model. Expert Syst Appl 208:118128. https:\/\/doi.org\/10.1016\/j.eswa.2022.118128","journal-title":"Expert Syst Appl"},{"key":"1400_CR15","doi-asserted-by":"publisher","first-page":"109584","DOI":"10.1016\/j.knosys.2022.109584","volume":"254","author":"P Zeng","year":"2022","unstructured":"Zeng P, Hu G, Zhou X, Li S, Liu P, Liu S (2022) Muformer: a long sequence time-series forecasting model based on modified multi-head attention. Knowl-Based Syst 254:109584. https:\/\/doi.org\/10.1016\/j.knosys.2022.109584","journal-title":"Knowl-Based Syst"},{"issue":"15","key":"1400_CR16","doi-asserted-by":"publisher","first-page":"1794","DOI":"10.3390\/math9151794","volume":"9","author":"E Ramos-P\u00e9rez","year":"2021","unstructured":"Ramos-P\u00e9rez E, Alonso-Gonz\u00e1lez PJ, N\u00fa\u00f1ez-Vel\u00e1zquez JJ (2021) Multi-transformer: a new neural network-based architecture for forecasting s & p volatility. Mathematics 9(15):1794. https:\/\/doi.org\/10.3390\/math9151794","journal-title":"Mathematics"},{"issue":"21","key":"1400_CR17","doi-asserted-by":"publisher","first-page":"30093","DOI":"10.1007\/s11042-022-11908-1","volume":"81","author":"J Chen","year":"2022","unstructured":"Chen J, Chen T, Shen M, Shi Y, Wang D, Zhang X (2022) Gated three-tower transformer for text-driven stock market prediction. Multimed Tools Appl 81(21):30093\u201330119. https:\/\/doi.org\/10.1007\/s11042-022-11908-1","journal-title":"Multimed Tools Appl"},{"key":"1400_CR18","volume-title":"Advances in financial machine learning","author":"ML Prado","year":"2018","unstructured":"Prado ML (2018) Advances in financial machine learning, 1st edn. Wiley, Hoboken","edition":"1"},{"key":"1400_CR19","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2020.0209","author":"B Lim","year":"2021","unstructured":"Lim B, Zohren S (2021) Time-series forecasting with deep learning: a survey. Philos Trans R Soc A. https:\/\/doi.org\/10.1098\/rsta.2020.0209","journal-title":"Philos Trans R Soc A"},{"key":"1400_CR20","volume-title":"Deep learning","author":"I Goodfellow","year":"2016","unstructured":"Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge"},{"key":"1400_CR21","volume-title":"Artificial intelligence: a modern approach, global edition","author":"S Russell","year":"2021","unstructured":"Russell S, Norvig P (2021) Artificial intelligence: a modern approach, global edition, 4th edn. Pearson, Harlow","edition":"4"},{"key":"1400_CR22","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1016\/j.neucom.2020.07.061","volume":"415","author":"L Yang","year":"2020","unstructured":"Yang L, Shami A (2020) On hyperparameter optimization of machine learning algorithms: theory and practice. Neurocomputing 415:295\u2013316. https:\/\/doi.org\/10.1016\/j.neucom.2020.07.061","journal-title":"Neurocomputing"},{"key":"1400_CR23","volume-title":"A random walk down Wall street: the time-tested strategy for successful investing","author":"BG Malkiel","year":"2023","unstructured":"Malkiel BG (2023) A random walk down Wall street: the time-tested strategy for successful investing, 50th edn. W.W. Norton and Company, New York","edition":"50"},{"key":"1400_CR24","series-title":"Springer texts in statistics","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-29854-2","volume-title":"Introduction to time series and forecasting","author":"PJ Brockwell","year":"2016","unstructured":"Brockwell PJ, Davis RA (2016) Introduction to time series and forecasting. Springer texts in statistics. Springer, New York. https:\/\/doi.org\/10.1007\/978-3-319-29854-2"},{"key":"1400_CR25","volume-title":"Forecasting: principles and practice","author":"R Hyndman","year":"2021","unstructured":"Hyndman R, Athanasopoulos G (2021) Forecasting: principles and practice, 3rd edn. OTexts, Melbourne","edition":"3"},{"issue":"6","key":"1400_CR26","doi-asserted-by":"publisher","first-page":"964","DOI":"10.1086\/644762","volume":"36","author":"P Raghubir","year":"2010","unstructured":"Raghubir P, Das SR (2010) The long and short of it: why are stocks with shorter runs preferred? J Consum Res 36(6):964\u2013982. https:\/\/doi.org\/10.1086\/644762","journal-title":"J Consum Res"},{"key":"1400_CR27","volume-title":"Nonparametric statistical methods","author":"M Hollander","year":"2013","unstructured":"Hollander M, Wolfe DA, Chicken E (2013) Nonparametric statistical methods, 3rd edn. Wiley, Hoboken","edition":"3"},{"issue":"3","key":"1400_CR28","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1177\/1536867X0900900307","volume":"9","author":"SJ Goerg","year":"2009","unstructured":"Goerg SJ, Kaiser J (2009) Nonparametric testing of distributions: the Epps\u2013Singleton two-sample test using the empirical characteristic function. Stata J 9(3):454\u2013465. https:\/\/doi.org\/10.1177\/1536867X0900900307","journal-title":"Stata J"},{"issue":"2","key":"1400_CR29","doi-asserted-by":"publisher","first-page":"802","DOI":"10.1111\/sjos.12544","volume":"49","author":"R Erlemann","year":"2022","unstructured":"Erlemann R, Lockhart R, Yao R (2022) Cram\u00e9r\u2013von Mises tests for change points. Scand J Stat 49(2):802\u2013830. https:\/\/doi.org\/10.1111\/sjos.12544","journal-title":"Scand J Stat"},{"key":"1400_CR30","doi-asserted-by":"publisher","DOI":"10.1017\/9781108684163","volume-title":"Mining of massive datasets","author":"J Leskovec","year":"2020","unstructured":"Leskovec J, Rajaraman A, Ullman JD (2020) Mining of massive datasets, 3rd edn. Cambridge University Press, Cambridge","edition":"3"},{"key":"1400_CR31","doi-asserted-by":"publisher","unstructured":"Ong E-J, Bober M (2016) Improved hamming distance search using variable length hashing. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, NV, pp 2000\u20132008. https:\/\/doi.org\/10.1109\/cvpr.2016.220","DOI":"10.1109\/cvpr.2016.220"},{"key":"1400_CR32","doi-asserted-by":"publisher","unstructured":"Wen R, Torkkola K, Narayanaswamy B, Madeka D (2017) A multi-horizon quantile recurrent forecaster. In: NIPS\u201917: proceedings of the 31st international conference on neural information processing systems. Curran Associates Inc., New York. https:\/\/doi.org\/10.48550\/arXiv.1711.11053","DOI":"10.48550\/arXiv.1711.11053"},{"key":"1400_CR33","unstructured":"Armbrust M, Ghodsi A, Xin R, Zaharia M (2021) Lakehouse: a new generation of open platforms that unify data warehousing and advanced analytics. In: 11th Conference on innovative data systems research, 8. https:\/\/researchr.org\/publication\/Zaharia0XA21"},{"key":"1400_CR34","unstructured":"Databricks: what is a Medallion architecture? databricks.com\/glossary\/medallion-architecture"},{"key":"1400_CR35","unstructured":"Preston H, Edwards T (2017) A practitioner\u2019s guide to reading VIX. S & P Global, New York"},{"key":"1400_CR36","doi-asserted-by":"publisher","unstructured":"Akiba T, Sano S, Yanase T, Ohta T, Koyama M (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. ACM, Anchorage, pp 2623\u20132631. https:\/\/doi.org\/10.1145\/3292500.3330701","DOI":"10.1145\/3292500.3330701"},{"issue":"1","key":"1400_CR37","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/1366\/1\/012130","volume":"1366","author":"FH Mustapa","year":"2019","unstructured":"Mustapa FH, Ismail MT (2019) Modelling and forecasting S &P 500 stock prices using hybrid Arima\u2013Garch model. J Phys Conf Ser 1366(1):012130. https:\/\/doi.org\/10.1088\/1742-6596\/1366\/1\/012130","journal-title":"J Phys Conf Ser"},{"key":"1400_CR38","doi-asserted-by":"publisher","unstructured":"Challu C, Olivares KG, Oreshkin BN, Garza F, Mergenthaler-Canseco M, Dubrawski A (2022) N-HiTS: neural hierarchical interpolation for time series forecasting. https:\/\/doi.org\/10.48550\/arXiv.2201.12886","DOI":"10.48550\/arXiv.2201.12886"},{"issue":"3","key":"1400_CR39","doi-asserted-by":"publisher","first-page":"1181","DOI":"10.1016\/j.ijforecast.2019.07.001","volume":"36","author":"D Salinas","year":"2020","unstructured":"Salinas D, Flunkert V, Gasthaus J, Januschowski T (2020) DeepAR: probabilistic forecasting with autoregressive recurrent networks. Int J Forecast 36(3):1181\u20131191. https:\/\/doi.org\/10.1016\/j.ijforecast.2019.07.001","journal-title":"Int J Forecast"},{"key":"1400_CR40","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.116583","volume":"195","author":"E Paquet","year":"2022","unstructured":"Paquet E, Soleymani F (2022) QuantumLeap: hybrid quantum neural network for financial predictions. Expert Syst Appl 195:116583. https:\/\/doi.org\/10.1016\/j.eswa.2022.116583","journal-title":"Expert Syst Appl"},{"key":"1400_CR41","unstructured":"Molnar C (2022) Interpretable machine learning: a guide for making black box models explainable. https:\/\/christophm.github.io\/interpretable-ml-book\/"},{"key":"1400_CR42","doi-asserted-by":"publisher","unstructured":"Schlegel U, Vo DL, Keim DA, Seebacher D (2021) TS-MULE: local interpretable model-agnostic explanations for time series forecast models. In: Machine learning and principles and practice of knowledge discovery in databases. Communications in computer and information science. Springer, New York, pp 5\u201314. https:\/\/doi.org\/10.1007\/978-3-030-93736-2_1","DOI":"10.1007\/978-3-030-93736-2_1"},{"key":"1400_CR43","doi-asserted-by":"publisher","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) \"Why should I trust you?\" Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. KDD \u201916. Association for Computing Machinery, New York, pp 1135\u20131144. https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"1400_CR44","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108102","volume":"120","author":"X Bai","year":"2021","unstructured":"Bai X, Wang X, Liu X, Liu Q, Song J, Sebe N, Kim B (2021) Explainable deep learning for efficient and robust pattern recognition: a survey of recent developments. Pattern Recognit 120:108102. https:\/\/doi.org\/10.1016\/j.patcog.2021.108102","journal-title":"Pattern Recognit"},{"key":"1400_CR45","doi-asserted-by":"publisher","unstructured":"Barry M, Bifet A, Billy J-L (2023) StreamAI: dealing with challenges of continual learning systems for serving AI in production. In: 2023 IEEE\/ACM 45th international conference on software engineering: software engineering in practice (ICSE-SEIP). IEEE, Melbourne, pp 134\u2013137. https:\/\/doi.org\/10.1109\/icse-seip58684.2023.00017","DOI":"10.1109\/icse-seip58684.2023.00017"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-024-01400-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-024-01400-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-024-01400-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,20]],"date-time":"2024-07-20T04:04:34Z","timestamp":1721448274000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-024-01400-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,8]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["1400"],"URL":"https:\/\/doi.org\/10.1007\/s40747-024-01400-8","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"type":"print","value":"2199-4536"},{"type":"electronic","value":"2198-6053"}],"subject":[],"published":{"date-parts":[[2024,4,8]]},"assertion":[{"value":"21 February 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all the authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}