{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,15]],"date-time":"2025-12-15T04:26:20Z","timestamp":1765772780159,"version":"3.37.3"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"13","license":[{"start":{"date-parts":[[2019,11,9]],"date-time":"2019-11-09T00:00:00Z","timestamp":1573257600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,11,9]],"date-time":"2019-11-09T00:00:00Z","timestamp":1573257600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior","doi-asserted-by":"publisher","award":["001"],"award-info":[{"award-number":["001"]}],"id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000857","name":"Loughborough University","doi-asserted-by":"publisher","award":["00"],"award-info":[{"award-number":["00"]}],"id":[{"id":"10.13039\/501100000857","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Soft Comput"],"published-print":{"date-parts":[[2020,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n<jats:p>Most information sources in the current technological world are generating data sequentially and rapidly, in the form of data streams. The evolving nature of processes may often cause changes in data distribution, also known as concept drift, which is difficult to detect and causes loss of accuracy in supervised learning algorithms. As a consequence, online machine learning algorithms that are able to update actively according to possible changes in the data distribution are required. Although many strategies have been developed to tackle this problem, most of them are designed for classification problems. Therefore, in the domain of regression problems, there is a need for the development of accurate algorithms with dynamic updating mechanisms that can operate in a computational time compatible with today\u2019s demanding market. In this article, the authors propose a new bagging ensemble approach based on neural network with random weights for online data stream regression. The proposed method improves the data prediction accuracy as well as minimises the required computational time compared to a recent algorithm for online data stream regression from literature. The experiments are carried out using four synthetic datasets to evaluate the algorithm\u2019s response to concept drift, along with four benchmark datasets from different industries. The results indicate improvement in data prediction accuracy, effectiveness in handling concept drift, and much faster updating times compared to the existing available approach. Additionally, the use of design of experiments as an effective tool for hyperparameter tuning is demonstrated.<\/jats:p>","DOI":"10.1007\/s00500-019-04499-x","type":"journal-article","created":{"date-parts":[[2019,11,9]],"date-time":"2019-11-09T11:02:36Z","timestamp":1573297356000},"page":"9835-9855","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["An ensemble based on neural networks with random weights for online data stream regression"],"prefix":"10.1007","volume":"24","author":[{"given":"Ricardo","family":"de Almeida","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yee Mey","family":"Goh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8782-3426","authenticated-orcid":false,"given":"Radmehr","family":"Monfared","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maria Teresinha Arns","family":"Steiner","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrew","family":"West","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2019,11,9]]},"reference":[{"key":"4499_CR1","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1016\/j.ins.2013.12.016","volume":"264","author":"M Alhamdoosh","year":"2014","unstructured":"Alhamdoosh M, Wang D (2014) Fast decorrelated neural network ensembles with random weights. Inf Sci 264:104\u2013117","journal-title":"Inf Sci"},{"key":"4499_CR2","doi-asserted-by":"crossref","unstructured":"Bifet A, Holmes G, Pfahringer B, Kirkby R, Gavald\u00e0 R (2009) New ensemble methods for evolving data streams. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining, pp 139\u2013148","DOI":"10.1145\/1557019.1557041"},{"key":"4499_CR3","first-page":"123","volume":"24","author":"L Breiman","year":"1996","unstructured":"Breiman L (1996) Bagging predictors. Mach Learn 24:123\u2013140","journal-title":"Mach Learn"},{"key":"4499_CR4","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45:5\u201332","journal-title":"Mach Learn"},{"key":"4499_CR5","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.inffus.2004.04.004","volume":"6","author":"G Brown","year":"2005","unstructured":"Brown G, Wyatt J, Harris H, Yao X (2005) Diversity creation methods: a survey and categorization. Inf Fusion 6:5\u201320","journal-title":"Inf Fusion"},{"key":"4499_CR6","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1080\/095400996116820","volume":"8","author":"R Bruce","year":"1996","unstructured":"Bruce R (1996) Ensemble learning using decorrelated neural networks. Connect Sci 8:373\u2013384","journal-title":"Connect Sci"},{"key":"4499_CR8","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.neucom.2017.08.040","volume":"275","author":"W Cao","year":"2018","unstructured":"Cao W, Wang X, Ming Z, Gao J (2018) A review on neural networks with random weights. Neurocomputing 275:278\u2013287","journal-title":"Neurocomputing"},{"issue":"20","key":"4499_CR9","doi-asserted-by":"publisher","first-page":"5919","DOI":"10.1007\/s00500-016-2269-9","volume":"21","author":"J Ding","year":"2017","unstructured":"Ding J, Wang H, Li C, Chai T, Wang J (2017) An online learning neural network ensemble with random weights for regression of sequential data stream. Soft Comput 21(20):5919\u20135937","journal-title":"Soft Comput"},{"issue":"10","key":"4499_CR10","doi-asserted-by":"publisher","first-page":"1517","DOI":"10.1109\/TNN.2011.2160459","volume":"22","author":"R Elwell","year":"2011","unstructured":"Elwell R, Polikar R (2011) Incremental learning of concept drift in nonstationary environments. IEEE Trans Neural Netw 22(10):1517\u20131531","journal-title":"IEEE Trans Neural Netw"},{"key":"4499_CR11","doi-asserted-by":"crossref","unstructured":"Fan W (2004) Systematic data selection to mine concept-drifting data streams. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining, pp 128\u2013137","DOI":"10.1145\/1014052.1014069"},{"key":"4499_CR12","doi-asserted-by":"publisher","first-page":"5895","DOI":"10.1016\/j.eswa.2013.05.001","volume":"40","author":"DM Farid","year":"2013","unstructured":"Farid DM, Zhang L, Hossain A, Rahman CM, Strachan R, Sexton G, Dahal K (2013) An adaptive ensemble classifier for mining concept drifting data streams. Expert Syst Appl 40:5895\u20135906","journal-title":"Expert Syst Appl"},{"issue":"1","key":"4499_CR13","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.is.2018.01.003","volume":"74","author":"CD Francescomarino","year":"2018","unstructured":"Francescomarino CD, Dumas M, Federici M, Ghidini C, Maggi FM, Rizzi W, Simonetto L (2018) Genetic algorithms for hyperparameter optimization in predictive business process monitoring. Inf Syst 74(1):67\u201383","journal-title":"Inf Syst"},{"key":"4499_CR14","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.inffus.2016.07.001","volume":"34","author":"PP G\u00e1llego","year":"2017","unstructured":"G\u00e1llego PP, Quevedo JR, Coz JJ (2017) Using ensembles for problems with characterizable changes in data distribution: a case study on quantification. Inf Fusion 34:87\u2013100","journal-title":"Inf Fusion"},{"issue":"6","key":"4499_CR15","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1109\/MIC.2008.119","volume":"12","author":"J Gao","year":"2008","unstructured":"Gao J, Ding B, Han J, Fan W, Yu PS (2008) Classifying data streams with skewed class distributions and concept drifts. IEEE Internet Comput 12(6):37\u201349","journal-title":"IEEE Internet Comput"},{"issue":"2","key":"4499_CR16","doi-asserted-by":"publisher","first-page":"23:1","DOI":"10.1145\/3054925","volume":"50","author":"HM Gomes","year":"2017","unstructured":"Gomes HM, Bardal JP, Enembreck F, Bifet A (2017) Survey on ensemble learning for data stream classification. ACM Comput Surv 50(2):23:1\u201323:36","journal-title":"ACM Comput Surv"},{"issue":"1","key":"4499_CR17","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","volume":"12","author":"AE Hoerl","year":"1970","unstructured":"Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55\u201367","journal-title":"Technometrics"},{"key":"4499_CR101","unstructured":"Huang G-B, Zhu Q-Y, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of the 2004 IEEE international joint conference on neural networks, vol 2, pp 985\u2013990"},{"issue":"4","key":"4499_CR18","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1007\/s13218-015-0381-0","volume":"29","author":"F Hutter","year":"2015","unstructured":"Hutter F, L\u00fccke J, Schmidt-Thieme L (2015) Beyond manual tuning of hyperparameters. KI \u2013 K\u00fcnstliche Intelligenz 29(4):329\u2013337","journal-title":"KI \u2013 K\u00fcnstliche Intelligenz"},{"key":"4499_CR19","first-page":"1","volume":"23","author":"E Ikonomovska","year":"2010","unstructured":"Ikonomovska E, Gama J, Dzeroski S (2010) Learning model trees from evolving data streams. Data Min Knowl Discov 23:1\u201341","journal-title":"Data Min Knowl Discov"},{"key":"4499_CR20","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1016\/j.neucom.2014.04.076","volume":"150","author":"E Ikonomovska","year":"2015","unstructured":"Ikonomovska E, Gama J, Dzeroski S (2015) Online tree-based ensembles and option trees for regression on evolving data streams. Neurocomputing 150:458\u2013470","journal-title":"Neurocomputing"},{"key":"4499_CR21","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1016\/j.future.2019.01.005","volume":"95","author":"AS Iwashita","year":"2019","unstructured":"Iwashita AS, Albuquerque VHC, Papa JP (2019) Learning concept drift with ensembles of optimum-path forest-based classifiers. Future Gener Comput Syst 95:198\u2013211","journal-title":"Future Gener Comput Syst"},{"issue":"5","key":"4499_CR22","doi-asserted-by":"publisher","first-page":"1288","DOI":"10.1002\/aic.12346","volume":"57","author":"P Kadlec","year":"2011","unstructured":"Kadlec P, Gabrys B (2011) Local learning-based adaptive soft sensor for catalyst activation prediction. Am Inst Chem Eng 57(5):1288\u20131301","journal-title":"Am Inst Chem Eng"},{"key":"4499_CR23","unstructured":"Kolter JZ, Maloof MA (2005) Using additive expert ensembles to cope with concept drift. In: Proceedings of the 22th ACM international conference on machine learning, pp 449\u2013456"},{"key":"4499_CR24","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1016\/j.inffus.2017.02.004","volume":"37","author":"B Krawczyk","year":"2017","unstructured":"Krawczyk B, Minku LL, Gama J, Stefanoswski J, Wozniak M (2017) Ensemble learning for data stream analysis: a survey. Inf Fusion 37:132\u2013156","journal-title":"Inf Fusion"},{"issue":"6","key":"4499_CR25","doi-asserted-by":"publisher","first-page":"1411","DOI":"10.1109\/TNN.2006.880583","volume":"17","author":"N-Y Liang","year":"2006","unstructured":"Liang N-Y, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411\u20131423","journal-title":"IEEE Trans Neural Netw"},{"key":"4499_CR26","doi-asserted-by":"publisher","first-page":"1399","DOI":"10.1016\/S0893-6080(99)00073-8","volume":"12","author":"Y Liu","year":"1999","unstructured":"Liu Y, Yao X (1999) Ensemble learning via negative correlation. Neural Netw 12:1399\u20131404","journal-title":"Neural Netw"},{"key":"4499_CR27","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1007\/s10462-012-9338-y","volume":"42","author":"S Masoudnia","year":"2014","unstructured":"Masoudnia S, Ebrahimpour R (2014) Mixture of experts: a literature survey. Artif Intell Rev 42:275\u2013293","journal-title":"Artif Intell Rev"},{"key":"4499_CR28","doi-asserted-by":"crossref","unstructured":"Masud MM, Gao J, Khan L, Han J (2008) A practical approach to classify evolving data streams: training with limited amount of labeled data. In: IEEE international conference on data mining, pp 929\u2013934","DOI":"10.1109\/ICDM.2008.152"},{"issue":"6","key":"4499_CR29","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1109\/TKDE.2010.61","volume":"23","author":"MM Masud","year":"2011","unstructured":"Masud MM, Gao J, Khan L, Han J, Thuraisingham B (2011) Classification and novel class detection in concept-drifting data streams under time constrains. IEEE Trans Knowl Data Eng 23(6):859\u2013874","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"4499_CR30","volume-title":"Design and analysis of experiments","author":"DC Montgomery","year":"2012","unstructured":"Montgomery DC (2012) Design and analysis of experiments, 8th edn. Wiley, Hoboken","edition":"8"},{"key":"4499_CR31","unstructured":"Oza NC, Russell S (2001) Online bagging and boosting. In: Proceedings of the eighth international workshop on artificial intelligence and statistics, pp 105\u2013112"},{"issue":"5","key":"4499_CR32","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1109\/2.144401","volume":"25","author":"Y-H Pao","year":"1992","unstructured":"Pao Y-H, Takefuji Y (1992) Functional-link net computing: theory, system architecture, and functionalities. IEEE Comput 25(5):76\u201379","journal-title":"IEEE Comput"},{"key":"4499_CR33","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/j.ins.2017.11.046","volume":"430\u2013431","author":"S Ren","year":"2018","unstructured":"Ren S, Liao B, Zhu W, Li K (2018) Knowledge-maximized ensemble algorithm for different types of concept drift. Inf Sci 430\u2013431:261\u2013281","journal-title":"Inf Sci"},{"key":"4499_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10462-009-9124-7","volume":"33","author":"L Rokach","year":"2010","unstructured":"Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33:1\u201339","journal-title":"Artif Intell Rev"},{"issue":"3\u20134","key":"4499_CR35","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1080\/095400996116820","volume":"8","author":"BE Rosen","year":"1996","unstructured":"Rosen BE (1996) Ensemble learning using decorrelated neural networks. Connect Sci 8(3\u20134):373\u2013393","journal-title":"Connect Sci"},{"key":"4499_CR36","first-page":"197","volume":"5","author":"RE Schapire","year":"1990","unstructured":"Schapire RE (1990) The strength of weak learnability. Mach Learn 5:197\u2013227","journal-title":"Mach Learn"},{"key":"4499_CR37","unstructured":"Schmidt WF, Kraaijveld MA, Duin RPW (1992) Feed forward neural networks with random weights. In: Proceedings of the 11th IAPR international conference on pattern recognition, 1992. vol II. Conference B: pattern recognition methodology and systems, IEEE, pp 1\u20134"},{"key":"4499_CR38","doi-asserted-by":"publisher","first-page":"392","DOI":"10.1016\/j.engappai.2014.10.003","volume":"37","author":"SG Soares","year":"2015","unstructured":"Soares SG, Ara\u00fajo R (2015a) An on-line weighted ensemble of regressor models to handle concept drifts. Eng Appl Artif Intell 37:392\u2013406","journal-title":"Eng Appl Artif Intell"},{"key":"4499_CR39","doi-asserted-by":"publisher","first-page":"2935","DOI":"10.1016\/j.eswa.2014.11.053","volume":"42","author":"SG Soares","year":"2015","unstructured":"Soares SG, Ara\u00fajo R (2015b) A dynamic and on-line ensemble regression for changing environments. Expert Syst Appl 42:2935\u20132948","journal-title":"Expert Syst Appl"},{"key":"4499_CR40","doi-asserted-by":"publisher","first-page":"1164","DOI":"10.1016\/j.eswa.2007.11.034","volume":"36","author":"C-J Tsai","year":"2009","unstructured":"Tsai C-J, Lee C-I, Yang W-P (2009) Mining decision rules on data streams in the presence of concept drifts. Expert Syst Appl 36:1164\u20131178","journal-title":"Expert Syst Appl"},{"key":"4499_CR100","unstructured":"Tsymbal A (2004) The problem of concept drift: definitions and related work. Technical report. Department of Computer Science, Trinity College, Dublin"},{"issue":"3\u20134","key":"4499_CR41","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1080\/095400996116839","volume":"8","author":"K Tumer","year":"1996","unstructured":"Tumer K, Ghosh J (1996) Error correlation and error reduction in ensemble classifiers. Connect Sci 8(3\u20134):385\u2013404","journal-title":"Connect Sci"},{"key":"4499_CR42","doi-asserted-by":"crossref","unstructured":"Wang H, Fan W, Yu PS, Han J (2003) Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, pp 226\u2013235","DOI":"10.1145\/956750.956778"},{"key":"4499_CR43","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","volume":"5","author":"DH Wolpert","year":"1992","unstructured":"Wolpert DH (1992) Stacked generalization. Neural Netw 5:241\u2013259","journal-title":"Neural Netw"},{"key":"4499_CR44","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.neucom.2014.06.092","volume":"165","author":"XC Yin","year":"2015","unstructured":"Yin XC, Huang K, Hao HW (2015) DE2: dynamic ensemble of ensembles for learning nonstationary data. Neurocomputing 165:14\u201322","journal-title":"Neurocomputing"},{"key":"4499_CR45","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1016\/j.ins.2016.01.039","volume":"364\u2013365","author":"L Zhang","year":"2016","unstructured":"Zhang L, Suganthan PN (2016) A survey of randomized algorithms for training neural networks. Inf Sci 364\u2013365:146\u2013155","journal-title":"Inf Sci"},{"key":"4499_CR46","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1007\/978-3-319-26989-4_4","volume-title":"Big data analysis: new algorithms for a new society","author":"I Zliobaite","year":"2016","unstructured":"Zliobaite I, Pechenizkiy M, Gama J (2016) An overview of concept drift applications. In: Japkowicz N, Stefanowski J (eds) Big data analysis: new algorithms for a new society. Springer, Berlin, pp 91\u2013114"}],"container-title":["Soft Computing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-019-04499-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s00500-019-04499-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-019-04499-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,8]],"date-time":"2020-11-08T00:34:18Z","timestamp":1604795658000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s00500-019-04499-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,9]]},"references-count":47,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2020,7]]}},"alternative-id":["4499"],"URL":"https:\/\/doi.org\/10.1007\/s00500-019-04499-x","relation":{},"ISSN":["1432-7643","1433-7479"],"issn-type":[{"type":"print","value":"1432-7643"},{"type":"electronic","value":"1433-7479"}],"subject":[],"published":{"date-parts":[[2019,11,9]]},"assertion":[{"value":"9 November 2019","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"All authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This article does not contain any studies with human participants performed by any of the authors.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}