{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T07:12:27Z","timestamp":1761808347575,"version":"build-2065373602"},"reference-count":55,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,12]],"date-time":"2020-01-12T00:00:00Z","timestamp":1578787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Recently, the concept of evaluating an unusually large learning effort of an adaptive system to detect novelties in the observed data was introduced. The present paper introduces a new measure of the learning effort of an adaptive system. The proposed method also uses adaptable parameters. Instead of a multi-scale enhanced approach, the generalized Pareto distribution is employed to estimate the probability of unusual updates, as well as for detecting novelties. This measure was successfully tested in various scenarios with (i) synthetic data, (ii) real time series datasets, and multiple adaptive filters and learning algorithms. The results of these experiments are presented.<\/jats:p>","DOI":"10.3390\/e22010093","type":"journal-article","created":{"date-parts":[[2020,1,13]],"date-time":"2020-01-13T04:05:51Z","timestamp":1578888351000},"page":"93","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Introduction to Extreme Seeking Entropy"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3538-7124","authenticated-orcid":false,"given":"Jan","family":"Vrba","sequence":"first","affiliation":[{"name":"Department of Computing and Control Engineering, Faculty of Chemical Engineering, University of Chemistry and Technology, 166 28 Prague, Czech Republic"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan","family":"Mare\u0161","sequence":"additional","affiliation":[{"name":"Department of Computing and Control Engineering, Faculty of Chemical Engineering, University of Chemistry and Technology, 166 28 Prague, Czech Republic"},{"name":"Department of Process Control, Faculty of Electrical Engineering and Informatics, University of Pardubice, 530 02 Pardubice, Czech Republic"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1016\/j.sigpro.2003.07.018","article-title":"Novelty detection: A review\u2014Part 1: Statistical approaches","volume":"83","author":"Markou","year":"2003","journal-title":"Signal Process."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1007\/s11265-010-0513-6","article-title":"Novelty detection with multivariate extreme value statistics","volume":"65","author":"Clifton","year":"2011","journal-title":"J. Signal Process. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hugueny, S., Clifton, D.A., and Tarassenko, L. (2010). Probabilistic patient monitoring with multivariate, multimodal extreme value theory. International Joint Conference on Biomedical Engineering Systems and Technologies, Springer.","DOI":"10.1007\/978-3-642-18472-7_16"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1007\/s11265-013-0835-2","article-title":"Extending the generalised Pareto distribution for novelty detection in high-dimensional spaces","volume":"74","author":"Clifton","year":"2014","journal-title":"J. Signal Process. Syst."},{"key":"ref_5","first-page":"6581","article-title":"One-class classification of point patterns of extremes","volume":"17","author":"Luca","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2499","DOI":"10.1016\/j.sigpro.2003.07.019","article-title":"Novelty detection: A review\u2014Part 2: Neural network based approaches","volume":"83","author":"Markou","year":"2003","journal-title":"Signal Process."},{"key":"ref_7","first-page":"157","article-title":"Novelty detection in learning systems","volume":"3","author":"Marsland","year":"2003","journal-title":"Neural Comput. Surv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1109\/9.847127","article-title":"Learning approach to nonlinear fault diagnosis: Detectability analysis","volume":"45","author":"Polycarpou","year":"2000","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.sigpro.2013.12.026","article-title":"A review of novelty detection","volume":"99","author":"Pimentel","year":"2014","journal-title":"Signal Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1186\/s13634-017-0507-7","article-title":"On the robustness of set-membership adaptive filtering algorithms","volume":"2017","author":"Yazdanpanah","year":"2017","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Diniz, P.S.R. (2013). Adaptive Filtering, Springer.","DOI":"10.1007\/978-1-4614-4106-9"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1109\/97.668945","article-title":"Set-membership filtering and a set-membership normalized LMS algorithm with an adaptive step size","volume":"5","author":"Gollamudi","year":"1998","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4159","DOI":"10.3390\/e15104159","article-title":"Learning Entropy: Multiscale Measure for Incremental Learning","volume":"15","author":"Bukovsky","year":"2013","journal-title":"Entropy"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bukovsky, I., Kinsner, W., and Homma, N. (2019). Learning Entropy as a Learning-Based Information Concept. Entropy, 21.","DOI":"10.3390\/e21020166"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.neucom.2018.04.069","article-title":"Concept drift robust adaptive novelty detection for data streams","volume":"309","author":"Cejnek","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bukovsky, I., Kinsner, W., and Bila, J. (2012, January 25\u201327). Multiscale analysis approach for novelty detection in adaptation plot. Proceedings of the Sensor Signal Processing for Defence (SSPD 2012), London, UK.","DOI":"10.1049\/ic.2012.0114"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.compbiomed.2018.09.030","article-title":"Early-warning of ARDS using novelty detection and data fusion","volume":"102","author":"Taoum","year":"2018","journal-title":"Comput. Biol. Med."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rad, N.M., van Laarhoven, T., Furlanello, C., and Marchiori, E. (2018). Novelty Detection using Deep Normative Modeling for IMU-Based Abnormal Movement Monitoring in Parkinson\u2019s Disease and Autism Spectrum Disorders. Sensors, 18.","DOI":"10.3390\/s18103533"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.compbiomed.2018.12.006","article-title":"Deep embeddings for novelty detection in myopathy","volume":"105","author":"Burlina","year":"2019","journal-title":"Comput. Biol. Med."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"012128","DOI":"10.1088\/1742-6596\/364\/1\/012128","article-title":"Application of novelty detection methods to health monitoring and typical fault diagnosis of a turbopump","volume":"364","author":"Hu","year":"2012","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_21","unstructured":"Surace, C., and Worden, K. (1998). A novelty detection method to diagnose damage in structures: An application to an offshore platform. The Eighth International Offshore and Polar Engineering Conference, International Society of Offshore and Polar Engineers."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bukovsky, I., Homma, N., Smetana, L., Rodriguez, R., Mironovova, M., and Vrana, S. (2010, January 7\u20139). Quadratic neural unit is a good compromise between linear models and neural networks for industrial applications. Proceedings of the 9th IEEE International Conference on Cognitive Informatics (ICCI\u201910), Beijing, China.","DOI":"10.1109\/COGINF.2010.5599677"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, M. (2013). Artificial Higher Order Neural Networks for Modeling and Simulation, IGI Global.","DOI":"10.4018\/978-1-4666-2175-6"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bukovsky, I., Voracek, J., Ichiji, K., and Noriyasu, H. (2017). Higher Order Neural Units for Efficient Adaptive Control of Weakly Nonlinear Systems. Proceedings of the 9th International Joint Conference on Computational Intelligence, SciTePress\u2014Science and Technology Publications.","DOI":"10.5220\/0006557301490157"},{"key":"ref_25","unstructured":"Haykin, S.S. (2005). Adaptive Filter Theory, Pearson Education India."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1109\/LSP.2003.821649","article-title":"A generalized normalized gradient descent algorithm","volume":"11","author":"Mandic","year":"2004","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/00401706.1969.10490657","article-title":"Procedures for detecting outlying observations in samples","volume":"11","author":"Grubbs","year":"1969","journal-title":"Technometrics"},{"key":"ref_28","unstructured":"Ma, J., and Perkins, S. (2003, January 20\u201324). Time-series novelty detection using one-class support vector machines. Proceedings of the International Joint Conference on Neural Networks, 2003, Portland, OR, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ma, J., and Perkins, S. (2003, January 24\u201327). Online novelty detection on temporal sequences. Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA.","DOI":"10.1145\/956750.956828"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Limpert, E., and Stahel, W.A. (2011). Problems with Using the Normal Distribution\u2014And Ways to Improve Quality and Efficiency of Data Analysis. PLoS ONE, 6.","DOI":"10.1371\/journal.pone.0021403"},{"key":"ref_31","first-page":"119","article-title":"Statistical inference using extreme order statistics","volume":"3","author":"Pickands","year":"1975","journal-title":"Ann. Stat."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"792","DOI":"10.1214\/aop\/1176996548","article-title":"Residual life time at great age","volume":"2","author":"Balkema","year":"1974","journal-title":"Ann. Probab."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Forbes, C., Evans, M., Hastings, N., and Peacock, B. (2010). Statistical Distributions, Wiley.","DOI":"10.1002\/9780470627242"},{"key":"ref_34","unstructured":"Coles, S. (2013). An Introduction to Statistical Modeling of Extreme Values, Springer."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Lee, H.J., and Roberts, S.J. (2008, January 8\u201311). On-line novelty detection using the Kalman filter and extreme value theory. Proceedings of the 2008 19th International Conference on Pattern Recognition, Tampa, FL, USA.","DOI":"10.1109\/ICPR.2008.4761918"},{"key":"ref_36","first-page":"33","article-title":"A review of extreme value threshold estimation and uncertainty quantification","volume":"10","author":"Scarrott","year":"2012","journal-title":"REVSTAT-Stat. J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1214\/aos\/1176346318","article-title":"Estimating the stable index \u03b1 in order to measure tail thickness: A critique","volume":"11","author":"DuMouchel","year":"1983","journal-title":"Ann. Stat."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1080\/0233188021000055345","article-title":"On optimising the estimation of high quantiles of a probability distribution","volume":"37","author":"Ferreira","year":"2003","journal-title":"Statistics"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/0927-5398(94)90004-3","article-title":"Testing the covariance stationarity of heavy-tailed time series: An overview of the theory with applications to several financial datasets","volume":"1","author":"Loretan","year":"1994","journal-title":"J. Empir. Financ."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1002\/j.2161-4296.2010.tb01766.x","article-title":"Detection of variance changes and mean value jumps in measurement noise for multipath mitigation in urban navigation","volume":"57","author":"Spangenberg","year":"2010","journal-title":"Navigation"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"L\u2019Ecuyer, P. (2017, January 3\u20136). History of uniform random number generation. Proceedings of the 2017 Winter Simulation Conference (WSC), Las Vegas, NV, USA.","DOI":"10.1109\/WSC.2017.8247790"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.engappai.2006.06.020","article-title":"Fault diagnosis using dynamic trend analysis: A review and recent developments","volume":"20","author":"Maurya","year":"2007","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Strogatz, S.H. (2018). Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering, CRC Press.","DOI":"10.1201\/9780429399640"},{"key":"ref_44","first-page":"2309","article-title":"Analysis of EEG signals using nonlinear dynamics and chaos: A review","volume":"9","year":"2015","journal-title":"Appl. Math. Inf. Sci."},{"key":"ref_45","unstructured":"Van Rossum, G., and Drake, F.L. (1995). Python Tutorial, Centrum voor Wiskunde en Informatica."},{"key":"ref_46","unstructured":"Oliphant, T.E. (2006). A Guide to NumPy, Trelgol Publishing USA. Available online: https:\/\/ecs.wgtn.ac.nz\/foswiki\/pub\/Support\/ManualPagesAndDocumentation\/numpybook.pdf."},{"key":"ref_47","unstructured":"Jones, E., Oliphant, T., and Peterson, P. (2020, January 11). SciPy: Open Source Scientific Tools for Python. Available online: https:\/\/www.bibsonomy.org\/bibtex\/21b37d2cc741af879d7958f2f7c23c420\/microcuts."},{"key":"ref_48","unstructured":"Cejnek, M. (2017). Padasip\u2014Open source library for adaptive signal processing in language Python. Studentsk\u00e1 Tv\u016fr\u010d\u00ed \u010dinnost 2017, Department of Instrumentation and Control Engineering."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: A 2D graphics environment","volume":"9","author":"Hunter","year":"2007","journal-title":"Comput. Sci. Eng."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1126\/science.267326","article-title":"Oscillation and chaos in physiological control systems","volume":"197","author":"Mackey","year":"1977","journal-title":"Science"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1111\/j.1467-842X.2006.00464.x","article-title":"Likelihood moment estimation for the generalized Pareto distribution","volume":"49","author":"Zhang","year":"2007","journal-title":"Aust. N. Z. J. Stat."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"7761","DOI":"10.1080\/03610918.2016.1249884","article-title":"Parameter estimation for generalized Pareto distribution by generalized probability weighted moment-equations","volume":"46","author":"Chen","year":"2017","journal-title":"Commun. Stat.-Simul. Comput."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Zhao, X., Zhang, Z., Cheng, W., and Zhang, P. (2019). A New Parameter Estimator for the Generalized Pareto Distribution under the Peaks over Threshold Framework. Mathematics, 7.","DOI":"10.3390\/math7050406"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","article-title":"The meaning and use of the area under a receiver operating characteristic (ROC) curve","volume":"143","author":"Hanley","year":"1982","journal-title":"Radiology"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","article-title":"The use of the area under the ROC curve in the evaluation of machine learning algorithms","volume":"30","author":"Bradley","year":"1997","journal-title":"Pattern Recognit."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/1\/93\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:29:25Z","timestamp":1760362165000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/1\/93"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,12]]},"references-count":55,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["e22010093"],"URL":"https:\/\/doi.org\/10.3390\/e22010093","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,1,12]]}}}