{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:24:12Z","timestamp":1740122652354,"version":"3.37.3"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"22","license":[{"start":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T00:00:00Z","timestamp":1692835200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T00:00:00Z","timestamp":1692835200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002241","name":"Japan Science and Technology Agency","doi-asserted-by":"publisher","award":["191400000190"],"award-info":[{"award-number":["191400000190"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2023,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We address the issue of detecting changes of models that lie behind a data stream. The model refers to an integer-valued structural information such as the number of free parameters in a parametric model. Specifically we are concerned with the problem of how we can detect signs of model changes earlier than they are actualized. To this end, we employ <jats:italic>continuous model selection<\/jats:italic> on the basis of the notion of <jats:italic>descriptive dimensionality<\/jats:italic>\u00a0(Ddim). It is a real-valued model dimensionality, which is designed for quantifying the model dimensionality in the model transition period. Continuous model selection is to determine the real-valued model dimensionality in terms of Ddim from a given data. We propose a novel methodology for detecting signs of model changes by tracking the rise-up\/descent of Ddim in a data stream. We apply this methodology to detecting signs of changes of the number of clusters in a Gaussian mixture model and those of the order in an auto regression model. With synthetic and real data sets, we empirically demonstrate its effectiveness by showing that it is able to visualize well how rapidly model dimensionality moves in the transition period and to raise early warning signals of model changes earlier than they are detected with existing methods.<\/jats:p>","DOI":"10.1007\/s10489-023-04780-5","type":"journal-article","created":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T05:01:54Z","timestamp":1692853314000},"page":"26454-26471","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Detecting signs of model change with continuous model selection based on descriptive dimensionality"],"prefix":"10.1007","volume":"53","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7370-9991","authenticated-orcid":false,"given":"Kenji","family":"Yamanishi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"So","family":"Hirai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,8,24]]},"reference":[{"key":"4780_CR1","unstructured":"Alquier P, Ridgway J,Chopin N (2016) \u201cOn the properties of variational approximations of Gibbs posteriors.\u201dJ Mach Learning Research 17,1\u201341"},{"key":"4780_CR2","doi-asserted-by":"crossref","unstructured":"Barron A, Cover T (1991) Minimum complexity density estimation. IEEE Trans Information Theory IT 37:1034\u20131054","DOI":"10.1109\/18.86996"},{"key":"4780_CR3","doi-asserted-by":"crossref","unstructured":"Davis RA, Lee T, Rodriguez-Yam G (2006) \u201cStructural break estimation for nonstationary time series models.\u201d Jr.of the Amer. Stat. Assoc. 101(473):22\u2013239","DOI":"10.1198\/016214505000000745"},{"key":"4780_CR4","doi-asserted-by":"crossref","unstructured":"Ding J, Tarokh V,Yang Y (2018) Model selection techniques: an overview. In: IEEE Signal Processing Magazine Vol. 35, Issue: 6, pp 16\u201334 Nov 2018","DOI":"10.1109\/MSP.2018.2867638"},{"key":"4780_CR5","doi-asserted-by":"crossref","unstructured":"Ding J, Zhou J,Tarokh V (2019) Asymptotically optimal prediction for time-varying data generating processes. In: IEEE Transactions on Information Theory Vol. 65, Issue: 5, pp 3034\u20133067 May 2019","DOI":"10.1109\/TIT.2018.2882819"},{"key":"4780_CR6","unstructured":"Dheeru D, Taniskidou EK (2017) UCI machine learning repository"},{"issue":"4","key":"4780_CR7","doi-asserted-by":"publisher","first-page":"1306","DOI":"10.1214\/aop\/1176991978","volume":"15","author":"RM Dudley","year":"1987","unstructured":"Dudley RM (1987) Universal Donsker classes and metric entropy. Annals of Probability 15(4):1306\u20131326","journal-title":"Annals of Probability"},{"key":"4780_CR8","doi-asserted-by":"crossref","unstructured":"Erven T, Grunwald P,Rooij S (2012) \u201cCatching up by switching sooner: a predictive approach to adaptive estimation with an application to the AIC-BIC dilemma.\" Jr.Royal Stat Soc Ser B Vol. 74, No.Issue 3 pp 361\u2013417","DOI":"10.1111\/j.1467-9868.2011.01025.x"},{"key":"4780_CR9","doi-asserted-by":"publisher","first-page":"1304","DOI":"10.1515\/zna-1982-1117","volume":"31","author":"JD Farmer","year":"1982","unstructured":"Farmer JD (1982) Information dimension and the probabilistic structure of chaos. Z. Naturforsch A 31:1304\u20131325","journal-title":"Z. Naturforsch A"},{"issue":"4","key":"4780_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2523813","volume":"46","author":"J Gama","year":"2014","unstructured":"Gama J, Zlibait I, Bifet A, Pechenizkiy M, Bouchachia A (2014) A survey on concept drift adaptation. ACM Computing Survey 46(4):1\u201337","journal-title":"ACM Computing Survey"},{"key":"4780_CR11","doi-asserted-by":"crossref","unstructured":"Grunwald PD (2007) The minimum description length principle, MIT press","DOI":"10.7551\/mitpress\/4643.001.0001"},{"key":"4780_CR12","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/0890-5401(92)90010-D","volume":"100","author":"D Haussler","year":"1992","unstructured":"Haussler D (1992) Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation 100:78\u2013150","journal-title":"Information and Computation"},{"key":"4780_CR13","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1023\/A:1007424614876","volume":"32","author":"M Herbster","year":"1998","unstructured":"Herbster M, Warmuth M (1998) Tracking the best experts. Machine Learning 32:151\u2013178","journal-title":"Machine Learning"},{"key":"4780_CR14","doi-asserted-by":"crossref","unstructured":"Hirai S, Yamanishi K (2012) \u201cDetecting changes of clustering structures using normalized maximum likelihood coding.\" In: Proceedings of ACM International Conference on Knowedge Discovery and Data Mining (KDD\u201912) pp 343\u2013351","DOI":"10.1145\/2339530.2339587"},{"issue":"10","key":"4780_CR15","doi-asserted-by":"publisher","first-page":"6827","DOI":"10.1109\/TIT.2019.2915237","volume":"65","author":"S Hirai","year":"2019","unstructured":"Hirai S, Yamanishi K (2019) Correction to efficient computation of normalized maximum likelihood codes for Gaussian mixture models with its applications to clustering. IEEE Transactions on Information Theory 65(10):6827\u20136828","journal-title":"IEEE Transactions on Information Theory"},{"key":"4780_CR16","doi-asserted-by":"crossref","unstructured":"Hirai S, Yamanishi K (2018) \u201cDetecting structural uncertainty with structural entropy.\u201d In: Proceedings of 2018 IEEE International Conference on BigData(BigData\u201918) pp 26\u201335","DOI":"10.1109\/BigData.2018.8622283"},{"key":"4780_CR17","doi-asserted-by":"crossref","unstructured":"Huang DTJ, Koh YS, Dobbie G,Pears R (2014) \u201cDetecting volatility shift in data streams.\u201d In: Proceedings of 2014 IEEE International Conference on Data Mining (ICDM\u201914), pp 863\u2013868","DOI":"10.1109\/ICDM.2014.50"},{"issue":"17","key":"4780_CR18","first-page":"277","volume":"2","author":"AN Kolmogorov","year":"1961","unstructured":"Kolmogorov AN, Tihomirov VM (1961) $$\\epsilon $$-entropy and $$\\epsilon $$-capacity of sets in functional spaces. Amer. Math. Sot. Trunsl. Ser. 2(17):277\u2013364","journal-title":"Amer. Math. Sot. Trunsl. Ser."},{"key":"4780_CR19","doi-asserted-by":"publisher","DOI":"10.1002\/0471721182","volume-title":"Finite Mixture Models","author":"G MacLachlan","year":"2000","unstructured":"MacLachlan G, Peel D (2000) Finite Mixture Models. John Wiley and Sons"},{"key":"4780_CR20","volume-title":"The Fractal Geometry of Nature","author":"BB Mandelbrot","year":"1982","unstructured":"Mandelbrot BB (1982) The Fractal Geometry of Nature. Freeman, W.H"},{"key":"4780_CR21","first-page":"562","volume":"11","author":"I Ntoutsi","year":"2011","unstructured":"Ntoutsi I, Spiliopoulou MY (2011) \u201cTheodoridis:FINGERPRINT: Summarizing cluster evolution in dynamic environments\u2019.Prceedings of Computational Science and Its Applications-ICCSA 2011. Part II 11:562\u2013577","journal-title":"Part II"},{"key":"4780_CR22","doi-asserted-by":"crossref","unstructured":"Ohsawa Y (2018) Graph-based entropy for detecting explanatory signs of changes in market. In: The Review of Social Network Strategies Vol. 12, 2, pp 183\u2013203","DOI":"10.1007\/s12626-018-0023-8"},{"key":"4780_CR23","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-5254-2","volume-title":"Convergence of Stochastic Processes","author":"D Pollard","year":"1984","unstructured":"Pollard D (1984) Convergence of Stochastic Processes. Springer-Verlag, Berlin\/NewYork"},{"key":"4780_CR24","doi-asserted-by":"crossref","unstructured":"Rissanen J (1978) Modeling by shortest description length. Automatica 14(5):465\u2013471","DOI":"10.1016\/0005-1098(78)90005-5"},{"key":"4780_CR25","doi-asserted-by":"crossref","unstructured":"Rissanen J (2012) Optimal Estimation of Parameters. Cambridge","DOI":"10.1017\/CBO9780511791635"},{"key":"4780_CR26","unstructured":"Shtarkov YM (1987) Universal sequential coding of single messages. Problem Peredachi Informatsii 23(3):3\u201317"},{"key":"4780_CR27","doi-asserted-by":"crossref","unstructured":"Song M, Wang H (2005) \u201cHighly efficient incremental estimation of Gaussian mixture models for online data stream clustering.\u201d Intelligent Computing: Theory and Applications III 5803:174\u2013183","DOI":"10.1117\/12.601724"},{"key":"4780_CR28","volume-title":"Estimation of Dependences Based on Empirical Data","author":"VN Vapnik","year":"1982","unstructured":"Vapnik VN (1982) Estimation of Dependences Based on Empirical Data. Springer Verlag, New York"},{"key":"4780_CR29","doi-asserted-by":"crossref","unstructured":"Xuan X, Murphy K (2007) \u201cModeling changing dependency structure in multivariate time series.\u201d Proceedings of the 24th International Conference on Machine learning pp 1055\u20131062 ACM","DOI":"10.1145\/1273496.1273629"},{"key":"4780_CR30","doi-asserted-by":"crossref","unstructured":"Yamanishi K (1992) \u201cA learning criterion for stochastic rules.\u201d Machine Learning Volume 9, Issue 2-3, pp 165-203 July 1992","DOI":"10.1007\/BF00992676"},{"key":"4780_CR31","unstructured":"Yamanishi K (2019) \u201cDescriptive dimensionality with its characterization of MDL-based learning and change detection.\u201d arXiv:1910.11540"},{"key":"4780_CR32","doi-asserted-by":"crossref","unstructured":"Yamanishi K, Hirai S (2023)\u201cDetecting signs of model change with continuous model selection based on descriptive dimensionality.\u201d arXiv:2302.12127","DOI":"10.1007\/s10489-023-04780-5"},{"key":"4780_CR33","doi-asserted-by":"crossref","unstructured":"Yamanishi K, Fukushima S (2018) Model change detection with the MDL principle. IEEE Transactions on Information Theory 64(9):6115\u20136126","DOI":"10.1109\/TIT.2018.2852747"},{"issue":"6","key":"4780_CR34","doi-asserted-by":"publisher","first-page":"2180","DOI":"10.1109\/TIT.2007.896890","volume":"53","author":"K Yamanishi","year":"2007","unstructured":"Yamanishi K, Maruyama Y (2007) Dynamic model selection with its applications to novelty detection. IEEE Transactions on Information Theory 53(6):2180\u20132189","journal-title":"IEEE Transactions on Information Theory"},{"key":"4780_CR35","doi-asserted-by":"crossref","unstructured":"Yamanishi K, Maruyama Y (2005) \u201cDynamic syslog mining for network failure monitoring.\" In: Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD\u201905) pp 499\u2013508","DOI":"10.1145\/1081870.1081927"},{"key":"4780_CR36","doi-asserted-by":"crossref","unstructured":"Yamanishi K, Miyaguchi K (2016) \u201cDetecting gradual changes from data stream using MDL change statistics.\" In: Proceedings of 2016 IEEE International Conference on BigData (BigData\u201916) pp 156\u2013163","DOI":"10.1109\/BigData.2016.7840601"},{"key":"4780_CR37","doi-asserted-by":"crossref","unstructured":"Yamanishi K, Xu L, Yuki R, Fukushima S, Lin C (2021) Change sign detection with differential MDL change statistics and its applications to COVID-19 pandemic analys. Scientific Reports 11:19795","DOI":"10.1038\/s41598-021-98781-4"},{"key":"4780_CR38","unstructured":"https:\/\/github.com\/conf-post-01\/Ddim.git"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-04780-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-023-04780-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-04780-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,25]],"date-time":"2023-10-25T23:06:54Z","timestamp":1698275214000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-023-04780-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,24]]},"references-count":38,"journal-issue":{"issue":"22","published-print":{"date-parts":[[2023,11]]}},"alternative-id":["4780"],"URL":"https:\/\/doi.org\/10.1007\/s10489-023-04780-5","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"type":"print","value":"0924-669X"},{"type":"electronic","value":"1573-7497"}],"subject":[],"published":{"date-parts":[[2023,8,24]]},"assertion":[{"value":"11 June 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 August 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}