{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T13:13:44Z","timestamp":1780492424372,"version":"3.54.1"},"reference-count":42,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:00:00Z","timestamp":1666051200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"DARPA SAIL-ON Contracts","award":["# HR001120C0055"],"award-info":[{"award-number":["# HR001120C0055"]}]},{"name":"DARPA SAIL-ON Contracts","award":["# W911NF2020004"],"award-info":[{"award-number":["# W911NF2020004"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Algorithms for automated novelty detection and management are of growing interest but must address the inherent uncertainty from variations in non-novel environments while detecting the changes from the novelty. This paper expands on a recent unified framework to develop an operational theory for novelty that includes multiple (sub)types of novelty. As an example, this paper explores the problem of multi-type novelty detection in a 3D version of CartPole, wherein the cart Weibull-Open-World control-agent (WOW-agent) is confronted by different sub-types\/levels of novelty from multiple independent agents moving in the environment. The WOW-agent must balance the pole and detect and characterize the novelties while adapting to maintain that balance. The approach develops static, dynamic, and prediction-error measures of dissimilarity to address different signals\/sources of novelty. The WOW-agent uses the Extreme Value Theory, applied per dimension of the dissimilarity measures, to detect outliers and combines different dimensions to characterize the novelty. In blind\/sequestered testing, the system detects nearly 100% of the non-nuisance novelties, detects many nuisance novelties, and shows it is better than novelty detection using a Gaussian-based approach. We also show the WOW-agent\u2019s lookahead collision avoiding control is significantly better than a baseline Deep-Q-learning Networktrained controller.<\/jats:p>","DOI":"10.3390\/a15100381","type":"journal-article","created":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T21:18:02Z","timestamp":1666127882000},"page":"381","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Weibull-Open-World (WOW) Multi-Type Novelty Detection in CartPole3D"],"prefix":"10.3390","volume":"15","author":[{"given":"Terrance E.","family":"Boult","sequence":"first","affiliation":[{"name":"VAST Lab, University of Colorado Colorado Springs, Colorado Springs, CO 80918, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1214-4242","authenticated-orcid":false,"given":"Nicolas M.","family":"Windesheim","sequence":"additional","affiliation":[{"name":"VAST Lab, University of Colorado Colorado Springs, Colorado Springs, CO 80918, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven","family":"Zhou","sequence":"additional","affiliation":[{"name":"Cheyenne Mountain High School, Colorado Springs, CO 80906, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Pereyda","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, Washington State University, Pullman, WA 642752, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6586-3144","authenticated-orcid":false,"given":"Lawrence B.","family":"Holder","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, Washington State University, Pullman, WA 642752, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,18]]},"reference":[{"key":"ref_1","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). OpenAI Gym. arXiv."},{"key":"ref_2","unstructured":"Xiao, T., Jang, E., Kalashnikov, D., Levine, S., Ibarz, J., Hausman, K., and Herzog, A. (2020). Thinking while moving: Deep reinforcement learning with concurrent control. arXiv."},{"key":"ref_3","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Boult, T., Grabowicz, P., Prijatelj, D., Stern, R., Holder, L., Alspector, J., Jafarzadeh, M., Ahmad, T., Dhamija, A., and Li, C. (2021, January 2\u20139). Towards a Unifying Framework for Formal Theories of Novelty. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually.","DOI":"10.1609\/aaai.v35i17.17766"},{"key":"ref_5","unstructured":"Coumans, E., and Bai, Y. (2021, January 08). PyBullet, a Python Module for Physics Simulation for Games, Robotics and Machine Learning. Available online: http:\/\/pybullet.org."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_7","unstructured":"Kumar, S. (2020). Balancing a CartPole System with Reinforcement Learning\u2013A Tutorial. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Jafarzadeh, M., Ahmad, T., Dhamija, A.R., Li, C., Cruz, S., and Boult, T.E. (2021, January 3\u20138). Automatic Open-World Reliability Assessment. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV48630.2021.00203"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kotz, S., and Nadarajah, S. (2001). Extreme Value Distributions: Theory and Applications, World Scientific Publishing Co.","DOI":"10.1142\/9781860944024"},{"key":"ref_10","unstructured":"Carpentier, A., and Valko, M. (2014, January 8\u201313). Extreme bandits. Proceedings of the NIPS, Montreal, QC, Canada."},{"key":"ref_11","first-page":"1","article-title":"Extreme value theory-based methods for visual recognition","volume":"7","author":"Scheirer","year":"2017","journal-title":"Synth. Lect. Comput. Vis."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gibert, X., Patel, V.M., and Chellappa, R. (2015, January 7\u201313). Sequential score adaptation with extreme value theory for robust railway track inspection. Proceedings of the IEEE International Conference on Computer Vision Workshops, Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.27"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1016\/0167-7152(91)90107-3","article-title":"On a basis for Peaks over Threshold modeling","volume":"12","author":"Leadbetter","year":"1991","journal-title":"Stat. & Probab. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Smith, R.L. (1984). Threshold methods for sample extremes. Statistical Extremes and Applications, Springer.","DOI":"10.1007\/978-94-017-3069-3_48"},{"key":"ref_15","unstructured":"Langley, P. (2020, January 7\u201312). Open-World Learning for Radically Autonomous Agents. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_16","unstructured":"Alspector, J. (2022, January 21\u201323). Representation Edit Distance as a Measure of Novelty. Proceedings of the AAAI Spring Symposium on Designing Artificial Intelligence for Open Worlds, Palo Alto, CA, USA."},{"key":"ref_17","unstructured":"Pereyda, C., and Holder, L. (2022, January 21\u201323). Measuring the Complexity of Domains Used to Evaluate AI Systems. Proceedings of the AAAI Spring Symposium on Designing Artificial Intelligence for Open Worlds, Palo Alto, CA, USA."},{"key":"ref_18","unstructured":"Doctor, K., Task, C., Kildebeck, E., Kejriwal, M., Holder, L., and Leong, R. (2022, January 21\u201323). Toward Defining Domain Complexity Measure Across Domains. Proceedings of the AAAI Spring Symposium on Designing Artificial Intelligence for Open Worlds, Palo Alto, CA, USA."},{"key":"ref_19","unstructured":"Sch\u00f6lkopf, B., Williamson, R.C., Smola, A., Shawe-Taylor, J., and Platt, J. (1999). Support vector method for novelty detection. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.sigpro.2013.12.026","article-title":"A review of novelty detection","volume":"99","author":"Pimentel","year":"2014","journal-title":"Signal Process."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/j.neucom.2013.12.002","article-title":"An experimental evaluation of novelty detection methods","volume":"135","author":"Ding","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"13093","DOI":"10.1073\/pnas.1814448115","article-title":"A neural data structure for novelty detection","volume":"115","author":"Dasgupta","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Abati, D., Porrello, A., Calderara, S., and Cucchiara, R. (2019, January 15\u201320). Latent space autoregression for novelty detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00057"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Perera, P., and Patel, V.M. (2019, January 15\u201320). Deep transfer learning for multiple class novelty detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01181"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rausch, A., Sedeh, A.M., and Zhang, M. (2021). Autoencoder-Based Semantic Novelty Detection: Towards Dependable AI-Based Systems. Appl. Sci., 11.","DOI":"10.3390\/app11219881"},{"key":"ref_26","first-page":"11839","article-title":"Csi: Novelty detection via contrastive learning on distributionally shifted instances","volume":"33","author":"Tack","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Perera, P., Nallapati, R., and Xiang, B. (2019, January 15\u201320). Ocgan: One-class novelty detection using gans with constrained latent representations. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00301"},{"key":"ref_28","unstructured":"Maddox, W., Garipov, T., Izmailov, P., Vetrov, D., and Wilson, A.G. (2018, January 7\u20139). Fast uncertainty estimates and bayesian model averaging of dnns. Proceedings of the Uncertainty in Deep Learning Workshop at UAI, Monterey, CA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zisselman, E., and Tamar, A. (2020, January 13\u201319). Deep residual flow for out of distribution detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01401"},{"key":"ref_30","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Deits, R., Koolen, T., and Tedrake, R. (2019, January 20\u201324). LVIS: Learning from value function intervals for contact-aware robot controllers. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794352"},{"key":"ref_32","unstructured":"Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., and Pathak, D. (2020, January 13\u201318). Planning to Explore via Self-Supervised World Models. Proceedings of the International Conference on Machine Learning (ICML), Virtual."},{"key":"ref_33","unstructured":"Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., de Las Casas, D., Budden, D., Abdolmaleki, A., Merel, J., and Lefrancq, A. (2018). DeepMind Control Suite. arXiv."},{"key":"ref_34","unstructured":"Conti, E., Madhavan, V., Such, F.P., Lehman, J., Stanley, K.O., and Clune, J. (2018). Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents. arXiv."},{"key":"ref_35","unstructured":"Peng, X., Balloch, J.C., and Riedl, M.O. (2021, January 9). Detecting and Adapting to Novelty in Games. Proceedings of the AAAI Workshop on Reinforcement Learning in Games, Virtual."},{"key":"ref_36","unstructured":"Muhammad, F., Sarathy, V., Tatiya, G., Goel, S., Gyawali, S., Guaman, M., Sinapov, J., and Scheutz, M. (2021, January 3\u20137). A Novelty-Centric Agent Architecture for Changing Worlds. Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS \u201921, Virtual."},{"key":"ref_37","unstructured":"Tatiya, G. (2022, August 09). Novel Gridworlds Environment for OpenAI Gym. Available online: https:\/\/github.com\/gtatiya\/gym-novel-gridworlds."},{"key":"ref_38","unstructured":"Voit, W., and Kildebeck, E. (2022, February 23). Polycraft World. Available online: https:\/\/www.polycraftworld.com."},{"key":"ref_39","unstructured":"Klenk, M., Piotrowski, W., Stern, R., Mohan, S., and de Kleer, J. (2020, January 26\u201328). Model-Based Novelty Adaptation for Open-World AI. Proceedings of the 31st International Workshop on Principles of Diagnosis, Virtual."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Gamage, C., Pinto, V., Xue, C., Stephenson, M., Zhang, P., and Renz, J. (2021, January 17\u201320). Novelty Generation Framework for AI Agents in Angry Birds Style Physics Games. Proceedings of the 2021 IEEE Conference on Games (CoG), Virtual.","DOI":"10.1109\/CoG52621.2021.9619160"},{"key":"ref_41","unstructured":"Xue, C., Pinto, V., Zhang, P., Gamage, C., Nikonova, E., and Renz, J. (2022, January 21\u201323). Science Birds Novelty: An Open-world Learning Test-bed for Physics Domains. Proceedings of the AAAI Spring Symposium on Designing AI for Open-World Novelty, Palo Alto, CA, USA."},{"key":"ref_42","unstructured":"Piotrowski, W., Stern, R., Klenk, M., Perez, A., Mohan, S., de Kleer, J., and Le, J. (2021, January 2\u201313). Playing Angry Birds with a Domain-Independent PDDL+ Planner. Proceedings of the 31st International Conference on Automated Planning Systems (Demo Track), Guangzhou, China."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/10\/381\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:56:10Z","timestamp":1760144170000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/10\/381"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,18]]},"references-count":42,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["a15100381"],"URL":"https:\/\/doi.org\/10.3390\/a15100381","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,18]]}}}