{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T05:42:38Z","timestamp":1781502158023,"version":"3.54.1"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,5,28]],"date-time":"2023-05-28T00:00:00Z","timestamp":1685232000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,28]],"date-time":"2023-05-28T00:00:00Z","timestamp":1685232000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Ensuring the optimal performance of power transformers is a laborious task in which the insulation system plays a vital role in decreasing their deterioration. The insulation system uses insulating oil to control temperature, as high temperatures can reduce the lifetime of the transformers and lead to expensive maintenance. Deep learning architectures have been demonstrated remarkable results in various fields. However, this improvement often comes at the cost of increased computing resources, which, in turn, increases the carbon footprint and hinders the optimization of architectures. In this study, we introduce a novel deep learning architecture that achieves a comparable efficacy to the best existing architectures in transformer oil temperature forecasting while improving efficiency. Effective forecasting can help prevent high temperatures and monitor the future condition of power transformers, thereby reducing unnecessary waste. To balance the inductive bias in our architecture, we propose the Smooth Residual Block, which divides the original problem into multiple subproblems to obtain different representations of the time series, collaboratively achieving the final forecasting. We applied our architecture to the Electricity Transformer datasets, which obtain transformer insulating oil temperature measures from two transformers in China. The results showed a 13% improvement in MSE and a 57% improvement in performance compared to the best current architectures, to the best of our knowledge. Moreover, we analyzed the architecture behavior to gain an intuitive understanding of the achieved solution.<\/jats:p>","DOI":"10.1186\/s40537-023-00745-0","type":"journal-article","created":{"date-parts":[[2023,5,28]],"date-time":"2023-05-28T13:01:31Z","timestamp":1685278891000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["A new deep learning architecture with inductive bias balance for transformer oil temperature forecasting"],"prefix":"10.1186","volume":"10","author":[{"given":"Manuel J.","family":"Jim\u00e9nez-Navarro","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mar\u00eda","family":"Mart\u00ednez-Ballesteros","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francisco","family":"Mart\u00ednez-\u00c1lvarez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gualberto","family":"Asencio-Cort\u00e9s","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,5,28]]},"reference":[{"issue":"13","key":"745_CR1","doi-asserted-by":"publisher","first-page":"4544","DOI":"10.3390\/s21134544","volume":"21","author":"A Rom\u00e1n-Portabales","year":"2021","unstructured":"Rom\u00e1n-Portabales A, L\u00f3pez-Nores M, Pazos-Arias JJ. Systematic review of electricity demand forecast using ANN-based machine learning algorithms. Sensors. 2021; 21(13):4544.","journal-title":"Sensors"},{"issue":"11","key":"745_CR2","doi-asserted-by":"publisher","first-page":"13162","DOI":"10.3390\/en81112361","volume":"8","author":"F Mart\u00ednez-\u00c1lvarez","year":"2015","unstructured":"Mart\u00ednez-\u00c1lvarez F, Troncoso A, Asencio-Cort\u00e9s G, Riquelme JC. A survey on data mining techniques applied to electricity-related time series forecasting. Energies. 2015; 8(11):13162\u201393.","journal-title":"Energies"},{"issue":"10","key":"745_CR3","doi-asserted-by":"publisher","first-page":"7697","DOI":"10.1016\/j.aej.2022.01.026","volume":"61","author":"VA Thiviyanathan","year":"2022","unstructured":"Thiviyanathan VA, Ker PJ, Leong YS, Abdullah F, Ismail A, Jamaludin ZM. Power transformer insulation system: a review on the reactions, fault detection, challenges and future prospects. Alex Eng J. 2022; 61(10):7697\u2013713.","journal-title":"Alex Eng J."},{"key":"745_CR4","doi-asserted-by":"crossref","unstructured":"Dursun K. Oil and winding temperature control in power transformers. In: Proceedings of the International Conference Power engineering, energy and electrical drives, 2013; pp. 1631\u20131639.","DOI":"10.1109\/PowerEng.2013.6635861"},{"key":"745_CR5","doi-asserted-by":"crossref","unstructured":"Beeram S.R, Kuchibhotla S. Time series analysis on univariate and multivariate variables: a comprehensive survey. In: Proceedings of the International Conference on Communication Software and Networks, 2021; pp. 119\u2013126.","DOI":"10.1007\/978-981-15-5397-4_13"},{"key":"745_CR6","doi-asserted-by":"crossref","unstructured":"Tabrizchi H, Razmara J, Mosavi A, Varkonyi-Koczy A.R. Deep learning applications for COVID-19: a brief review. In: Proceedings of the International Conference on Research and education: traditions and innovations, 2022; pp. 117\u2013130.","DOI":"10.1007\/978-981-19-0379-3_12"},{"key":"745_CR7","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-021-00444-8","author":"L Alzubaidi","year":"2021","unstructured":"Alzubaidi L, Zhang J, Humaidi AJ, Al-dujaili A, Duan Y, Al-Shamma O, Santamar\u00eda J, Fadhel MA, Al-Amidie M, Farhan L. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data, 2021; 8(1):53.","journal-title":"J Big Data"},{"key":"745_CR8","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1186\/s40537-021-00440-y","volume":"8","author":"M Al-Maleh","year":"2021","unstructured":"Al-Maleh M, Desouki S. Arabic text summarization using deep learning approach. J Big Data. 2021; 7(1):56.","journal-title":"J Big Data"},{"key":"745_CR9","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1186\/s40537-022-00561-y","volume":"9","author":"A Zewdu","year":"2022","unstructured":"Zewdu A, Yitagesu B. Part of speech tagging: a systematic review of deep learning and machine learning approaches. J Big Data. 2022; 9(1):10.","journal-title":"J Big Data"},{"key":"745_CR10","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1186\/s40537-021-00438-6","volume":"8","author":"M Aghdam","year":"2021","unstructured":"Aghdam M, Tabbakh SK, Chabok S, Kheirabadi M. Optimization of air traffic management efficiency based on deep learning enriched by the long short-term memory (LSTM) and extreme learning machine (ELM). J Big Data. 2021; 8(1):54.","journal-title":"J Big Data"},{"key":"745_CR11","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1186\/s40537-020-00333-6","volume":"7","author":"J Shen","year":"2020","unstructured":"Shen J, Shafiq M. Short-term stock market price trend prediction using a comprehensive deep learning system. J Big Data. 2020; 7(1):66.","journal-title":"J Big Data"},{"key":"745_CR12","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A.N, Kaiser L, Polosukhin I. Attention is all you need. In: Proceedings of the International Conference on Neural Information Processing Systems. 2017; pp. 6000\u20136010."},{"key":"745_CR13","doi-asserted-by":"crossref","unstructured":"Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W. Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the International Conference on Advancement of Artificial Intelligence. 2021; pp. 11106\u201311115.","DOI":"10.1609\/aaai.v35i12.17325"},{"key":"745_CR14","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2021.11.013","author":"S Makridakis","year":"2022","unstructured":"Makridakis S, Spiliotis E, Assimakopoulos V. M5 accuracy competition: results, findings, and conclusions. Int J Forecast. 2022; pp.\u00a01346\u20131364.","journal-title":"Int J Forecast"},{"key":"745_CR15","doi-asserted-by":"crossref","unstructured":"Sui J, Ling X, Xiang X, Zhang G, Zhang X, Transformer oil temperature prediction based on long and short-term memory networks. In: Proceedings of the International Conference on Big Data, 2021; pp. 6029\u20136031.","DOI":"10.1109\/BigData52589.2021.9671748"},{"key":"745_CR16","unstructured":"Oreshkin B.N, Carpov D, Chapados N, Bengio Y. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv. 2019."},{"key":"745_CR17","unstructured":"Liu M, Zeng A, Xu Z, Lai Q, Xu Q. Time Series is a Special Sequence: Forecasting with Sample Convolution and Interaction. arXiv. 2021."},{"key":"745_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpa.2021.100187","volume":"11","author":"J Klimek","year":"2022","unstructured":"Klimek J, Klimek J, Kra\u015bkiewicz W, Topolewski M. Query selector-efficient transformer with sparse attention. Softw Impacts. 2022; 11(1): 100187.","journal-title":"Softw Impacts"},{"key":"745_CR19","unstructured":"Du D, Su B, Wei Z. SCformer: segment correlation transformer for long sequence time series forecasting. In: Proceedings of International Conference on Learning Representations. 2022; pp. 1\u201312."},{"issue":"2","key":"745_CR20","first-page":"32","volume":"15","author":"J \u017dagar","year":"2019","unstructured":"\u017dagar J, Miheli\u010d J. Creation of attribute vectors from spectra and time-series data for prediction model development. IPSI Trans Internet Res. 2019; 15(2):1054.","journal-title":"IPSI Trans Internet Res"},{"key":"745_CR21","unstructured":"Woo G, Liu C, Sahoo D, Kumar A, Hoi S. CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. In: Proceedings of International Conference on Learning Representations. 2022; pp. 1\u201318."},{"key":"745_CR22","doi-asserted-by":"crossref","unstructured":"He K, Fan H, Wu Y, Xie S, Girshick R. Momentum contrast for unsupervised visual representation learning. In: Proceedings of International Conference on Computer Vision and Pattern Recognition, 2020; pp. 9726\u20139735.","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"745_CR23","unstructured":"Wu H, Xu J, Wang J, Long M. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In: Proceedings of International Conference on  Neural Information Processing Systems. 2021; pp. 1\u201312."},{"key":"745_CR24","doi-asserted-by":"crossref","unstructured":"Kosani\u0107 M, Milutinovi\u0107 V. A survey on mathematical aspects of machine learning in geophysics: The cases of weather forecast, wind energy, wave energy, oil and gas exploration. In: Proceedings of the International Conference on Embedded Computing. 2021; pp. 1\u20136.","DOI":"10.1109\/MECO52532.2021.9460245"},{"key":"745_CR25","unstructured":"Goyal A, Bengio Y. Inductive biases for deep learning of higher-level cognition. arXiv. 2020."},{"key":"745_CR26","unstructured":"LI X, Grandvalet Y, Davoine F. Explicit inductive bias for transfer learning with convolutional networks. In: Proceedings of the International Conference on Machine Learning. Machine Learning Research. 2018; pp. 2825\u20132834."},{"key":"745_CR27","unstructured":"Zeng A, Chen M, Zhang L, Xu Q. Are transformers effective for time series forecasting?. arXiv. 2022."},{"key":"745_CR28","doi-asserted-by":"crossref","unstructured":"Jim\u00e9nez-Navarro MJ, Mart\u00ednez-\u00c1lvarez F, Troncoso A, Cort\u00e9s GA. HLNet: A novel hierarchical deep neural network for time series forecasting. In: Proceedings of International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2021). 2022; pp. 717\u2013727.","DOI":"10.1007\/978-3-030-87869-6_68"},{"issue":"8","key":"745_CR29","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8):1735\u20131780.","journal-title":"Neural Comput"},{"key":"745_CR30","first-page":"2653","volume":"18","author":"A Benavoli","year":"2017","unstructured":"Benavoli A, Corani G, Demsar J, Zaffalon M. Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis. J Mach Learn Res. 2017; 18(77):2653\u201388.","journal-title":"J Mach Learn Res"}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-023-00745-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-023-00745-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-023-00745-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,28]],"date-time":"2023-05-28T13:01:38Z","timestamp":1685278898000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-023-00745-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,28]]},"references-count":30,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["745"],"URL":"https:\/\/doi.org\/10.1186\/s40537-023-00745-0","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,28]]},"assertion":[{"value":"20 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 May 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"80"}}