{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T21:39:52Z","timestamp":1784583592269,"version":"3.55.0"},"publisher-location":"Cham","reference-count":24,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783031333828","type":"print"},{"value":"9783031333835","type":"electronic"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,26]],"date-time":"2023-05-26T00:00:00Z","timestamp":1685059200000},"content-version":"vor","delay-in-days":145,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Dealing with an unbounded data stream involves overcoming the assumption that data is identically distributed and independent. A data stream can, in fact, exhibit temporal dependencies (i.e., be a time series), and data can change distribution over time (concept drift). The two problems are deeply discussed, and existing solutions address them separately: a joint solution is absent. In addition, learning multiple concepts implies remembering the past (a.k.a. avoiding catastrophic forgetting in Neural Networks\u2019 terminology). This work proposes Continuous Progressive Neural Networks (cPNN), a solution that tames concept drifts, handles temporal dependencies, and bypasses catastrophic forgetting. cPNN is a continuous version of Progressive Neural Networks, a methodology for remembering old concepts and transferring past knowledge to fit the new concepts quickly. We base our method on Recurrent Neural Networks and exploit the Stochastic Gradient Descent applied to data streams with temporal dependencies. Results of an ablation study show a quick adaptation of cPNN to new concepts and robustness to drifts.<\/jats:p>","DOI":"10.1007\/978-3-031-33383-5_26","type":"book-chapter","created":{"date-parts":[[2023,5,29]],"date-time":"2023-05-29T19:01:43Z","timestamp":1685386903000},"page":"328-340","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["cPNN: Continuous Progressive Neural Networks for\u00a0Evolving Streaming Time Series"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4210-6271","authenticated-orcid":false,"given":"Federico","family":"Giannini","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2768-3580","authenticated-orcid":false,"given":"Giacomo","family":"Ziffer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5176-5885","authenticated-orcid":false,"given":"Emanuele","family":"Della Valle","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,5,26]]},"reference":[{"issue":"2","key":"26_CR1","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1002\/sam.10151","volume":"5","author":"C Anagnostopoulos","year":"2012","unstructured":"Anagnostopoulos, C., Tasoulis, D.K., Adams, N.M., Pavlidis, N.G., Hand, D.J.: Online linear and quadratic discriminant analysis with adaptive forgetting for streaming classification. Stat. Anal. Data Min. 5(2), 139\u2013166 (2012)","journal-title":"Stat. Anal. Data Min."},{"key":"26_CR2","doi-asserted-by":"crossref","unstructured":"Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: PODS, pp. 1\u201316. ACM (2002)","DOI":"10.1145\/543613.543615"},{"key":"26_CR3","doi-asserted-by":"crossref","unstructured":"Bifet, A., Gavald\u00e0, R., Holmes, G., Pfahringer, B.: Machine learning for data streams: with practical examples in MOA. MIT press (2018)","DOI":"10.7551\/mitpress\/10654.001.0001"},{"key":"26_CR4","doi-asserted-by":"crossref","unstructured":"Cossu, A., Carta, A., Bacciu, D.: Continual learning with gated incremental memories for sequential data processing. In: IJCNN, pp. 1\u20138. IEEE (2020)","DOI":"10.1109\/IJCNN48605.2020.9207550"},{"key":"26_CR5","series-title":"Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence)","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1007\/978-3-540-28645-5_29","volume-title":"Advances in Artificial Intelligence","author":"J Gama","year":"2004","unstructured":"Gama, J., Medas, P., Castillo, G., Rodrigues, P.: Learning with drift detection. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 286\u2013295. Springer, Heidelberg (2004). https:\/\/doi.org\/10.1007\/978-3-540-28645-5_29"},{"key":"26_CR6","doi-asserted-by":"crossref","unstructured":"Gama, J., Sebasti\u00e3o, R., Rodrigues, P.P.: Issues in evaluation of stream learning algorithms. In: KDD, pp. 329\u2013338. ACM (2009)","DOI":"10.1145\/1557019.1557060"},{"key":"26_CR7","unstructured":"Goodfellow, I.J., Bengio, Y., Courville, A.C.: Deep Learning. Adaptive Computation and Machine Learning, MIT Press, Cambridge (2016)"},{"issue":"8","key":"26_CR8","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735\u20131780 (1997)","journal-title":"Neural Comput."},{"key":"26_CR9","doi-asserted-by":"crossref","unstructured":"Hulten, G., Spencer, L., Domingos, P.M.: Mining time-changing data streams. In: KDD, pp. 97\u2013106. ACM (2001)","DOI":"10.1145\/502512.502529"},{"issue":"7","key":"26_CR10","first-page":"3366","volume":"44","author":"MD Lange","year":"2022","unstructured":"Lange, M.D., et al.: A continual learning survey: defying forgetting in classification tasks. IEEE Trans. Pattern Anal. Mach. Intell. 44(7), 3366\u20133385 (2022)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"26_CR11","doi-asserted-by":"publisher","unstructured":"Lemos Neto, \u00c1.C., Coelho, R.A., Castro, C.L.: An incremental learning approach using long short-term memory neural networks. J. Control Autom. Electr. Syst. 1\u20139 (2022). https:\/\/doi.org\/10.1007\/s40313-021-00882-y","DOI":"10.1007\/s40313-021-00882-y"},{"key":"26_CR12","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1016\/j.inffus.2019.12.004","volume":"58","author":"T Lesort","year":"2020","unstructured":"Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., Rodr\u00edguez, N.D.: Continual learning for robotics: definition, framework, learning strategies, opportunities and challenges. Inf. Fusion 58, 52\u201368 (2020)","journal-title":"Inf. Fusion"},{"issue":"12","key":"26_CR13","first-page":"2346","volume":"31","author":"J Lu","year":"2019","unstructured":"Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., Zhang, G.: Learning under concept drift: a review. IEEE Trans. Knowl. Data Eng. 31(12), 2346\u20132363 (2019)","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"26_CR14","doi-asserted-by":"crossref","unstructured":"McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: the sequential learning problem. In: Psychology of Learning and Motivation, vol. 24, pp. 109\u2013165. Elsevier (1989)","DOI":"10.1016\/S0079-7421(08)60536-8"},{"key":"26_CR15","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/j.neucom.2019.01.037","volume":"337","author":"Q Meng","year":"2019","unstructured":"Meng, Q., Chen, W., Wang, Y., Ma, Z., Liu, T.: Convergence analysis of distributed stochastic gradient descent with shuffling. Neurocomputing 337, 46\u201357 (2019)","journal-title":"Neurocomputing"},{"issue":"10","key":"26_CR16","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","volume":"22","author":"SJ Pan","year":"2010","unstructured":"Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345\u20131359 (2010)","journal-title":"IEEE Trans. Knowl. Data Eng."},{"issue":"1865","key":"26_CR17","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1038\/072294b0","volume":"72","author":"K Pearson","year":"1905","unstructured":"Pearson, K.: The problem of the random walk. Nature 72(1865), 294\u2013294 (1905)","journal-title":"Nature"},{"key":"26_CR18","series-title":"Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence)","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1007\/978-3-030-61380-8_36","volume-title":"Intelligent Systems","author":"J Read","year":"2020","unstructured":"Read, J., Rios, R.A., Nogueira, T., de Mello, R.F.: Data streams are time series: challenging assumptions. In: Cerri, R., Prati, R.C. (eds.) BRACIS 2020. LNCS (LNAI), vol. 12320, pp. 529\u2013543. Springer, Cham (2020). https:\/\/doi.org\/10.1007\/978-3-030-61380-8_36"},{"key":"26_CR19","unstructured":"Rusu, A.A., et al.: Progressive neural networks. CoRR abs\/1606.04671 (2016)"},{"issue":"3","key":"26_CR20","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1007\/BF00116895","volume":"1","author":"JC Schlimmer","year":"1986","unstructured":"Schlimmer, J.C., Granger, R.H.: Incremental learning from noisy data. Mach. Learn. 1(3), 317\u2013354 (1986)","journal-title":"Mach. Learn."},{"key":"26_CR21","doi-asserted-by":"crossref","unstructured":"de Souza, V.M.A., dos Reis, D.M., Maletzke, A.G., Batista, G.E.A.P.A.: Challenges in benchmarking stream learning algorithms with real-world data. Data Min. Knowl. Discov. 34(6), 1805\u20131858 (2020)","DOI":"10.1007\/s10618-020-00698-5"},{"key":"26_CR22","doi-asserted-by":"crossref","unstructured":"Street, W.N., Kim, Y.: A streaming ensemble algorithm (SEA) for large-scale classification. In: KDD, pp. 377\u2013382. ACM (2001)","DOI":"10.1145\/502512.502568"},{"key":"26_CR23","unstructured":"Ziffer, G., Bernardo, A., Della Valle, E., Cerqueira, V., Bifet, A.: Towards time-evolving analytics: Online learning for time-dependent evolving data streams. Data Sci. 1\u201316 (in press)"},{"issue":"3","key":"26_CR24","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1007\/s10994-014-5441-4","volume":"98","author":"I Zliobaite","year":"2015","unstructured":"Zliobaite, I., Bifet, A., Read, J., Pfahringer, B., Holmes, G.: Evaluation methods and decision theory for classification of streaming data with temporal dependence. Mach. Learn. 98(3), 455\u2013482 (2015)","journal-title":"Mach. Learn."}],"container-title":["Lecture Notes in Computer Science","Advances in Knowledge Discovery and Data Mining"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-33383-5_26","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,13]],"date-time":"2024-03-13T20:09:49Z","timestamp":1710360589000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-33383-5_26"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031333828","9783031333835"],"references-count":24,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-33383-5_26","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"26 May 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"PAKDD","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Pacific-Asia Conference on Knowledge Discovery and Data Mining","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Osaka","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Japan","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2023","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"25 May 2023","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"28 May 2023","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"27","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"pakdd2023","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/pakdd2023.org\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Microsoft CMT","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"813","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"143","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"18% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3.5","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"10","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Yes","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}