{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T23:10:40Z","timestamp":1766790640617,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,6,10]],"date-time":"2022-06-10T00:00:00Z","timestamp":1654819200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004837","name":"MCIN\/AEI\/10.13039\/501100011033","doi-asserted-by":"publisher","award":["PID2020-115734RB-C21"],"award-info":[{"award-number":["PID2020-115734RB-C21"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Recent works have made significant progress in novelty detection, i.e., the problem of detecting samples of novel classes, never seen during training, while classifying those that belong to known classes. However, the only information this task provides about novel samples is that they are unknown. In this work, we leverage hierarchical taxonomies of classes to provide informative outputs for samples of novel classes. We predict their closest class in the taxonomy, i.e., its parent class. We address this problem, known as hierarchical novelty detection, by proposing a novel loss, namely Hierarchical Cosine Loss that is designed to learn class prototypes along with an embedding of discriminative features consistent with the taxonomy. We apply it to traffic sign recognition, where we predict the parent class semantics for new types of traffic signs. Our model beats state-of-the art approaches on two large scale traffic sign benchmarks, Mapillary Traffic Sign Dataset (MTSD) and Tsinghua-Tencent 100K (TT100K), and performs similarly on natural images benchmarks (AWA2, CUB). For TT100K and MTSD, our approach is able to detect novel samples at the correct nodes of the hierarchy with 81% and 36% of accuracy, respectively, at 80% known class accuracy.<\/jats:p>","DOI":"10.3390\/s22124389","type":"journal-article","created":{"date-parts":[[2022,6,13]],"date-time":"2022-06-13T02:01:44Z","timestamp":1655085704000},"page":"4389","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Hierarchical Novelty Detection for Traffic Sign Recognition"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0529-0871","authenticated-orcid":false,"given":"Idoia","family":"Ruiz","sequence":"first","affiliation":[{"name":"Computer Vision Center and Computer Science Department, Universitat Aut\u00f2noma de Barcelona, 08193 Bellaterra, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4554-199X","authenticated-orcid":false,"given":"Joan","family":"Serrat","sequence":"additional","affiliation":[{"name":"Computer Vision Center and Computer Science Department, Universitat Aut\u00f2noma de Barcelona, 08193 Bellaterra, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,10]]},"reference":[{"key":"ref_1","unstructured":"Masana, M., Ruiz, I., Serrat, J., van de Weijer, J., and Lopez, A.M. (2018, January 3\u20136). Metric Learning for Novelty and Anomaly Detection. Proceedings of the British Machine Vision Conference (BMVC), Newcastle, UK."},{"key":"ref_2","unstructured":"Lee, K., Lee, K., Lee, H., and Shin, J. (2018, January 3\u20138). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Montreal, QC, Canada."},{"key":"ref_3","unstructured":"Lee, K., Lee, H., Lee, K., and Shin, J. (May, January 30). Training confidence-calibrated classifiers for detecting out-of-distribution samples. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_4","unstructured":"Hendrycks, D., and Gimpel, K. (2017, January 24\u201326). A baseline for detecting misclassified and out-of-distribution examples in neural networks. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lee, K., Lee, K., Min, K., Zhang, Y., Shin, J., and Lee, H. (2018, January 18\u201322). Hierarchical Novelty Detection for Visual Object Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00114"},{"key":"ref_6","unstructured":"Boult, T.E., Cruz, S., Dhamija, A.R., Gunther, M., Henrydoss, J., and Scheirer, W.J. (February, January 27). Learning and the unknown: Surveying steps toward open world recognition. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_7","unstructured":"Denzler, J., Rodner, E., Bodesheim, P., and Freytag, A. (2013, January 3). Beyond the closed-world assumption: The importance of novelty detection and open set recognition. Proceedings of the German Conference on Pattern Recognition (GCPR) (Unsolved Problems in Pattern Recognition Workshop (UPPR)), Saarbr\u00fccken, Germany."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, F., Xiang, X., Cheng, J., and Yuille, A.L. (2017, January 23\u201327). Normface: L2 hypersphere embedding for face verification. Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, CA, USA.","DOI":"10.1145\/3123266.3123359"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2429","DOI":"10.1109\/TNNLS.2021.3104882","article-title":"Learning to classify with incremental new class","volume":"33","author":"Zhou","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wali, S.B., Abdullah, M.A., Hannan, M.A., Hussain, A., Samad, S.A., Ker, P.J., and Mansor, M.B. (2019). Vision-Based Traffic Sign Detection and Recognition Systems: Current Trends and Challenges. Sensors, 19.","DOI":"10.3390\/s19092093"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2233","DOI":"10.1007\/s00521-021-06526-1","article-title":"A real-time and high-precision method for small traffic-signs recognition","volume":"34","author":"Chen","year":"2022","journal-title":"Neural. Comput. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"17779","DOI":"10.1007\/s11042-022-12163-0","article-title":"Traffic sign recognition based on deep learning","volume":"81","author":"Zhu","year":"2022","journal-title":"Multimed. Tools. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Dewi, C., Chen, R.C., Jiang, X., and Yu, H. (2022). Deep convolutional neural network for enhancing traffic sign recognition developed on Yolo V4. Multimed. Tools. Appl., 1\u201325.","DOI":"10.1007\/s11042-022-12962-5"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3439950","article-title":"Deep Learning for Anomaly Detection: A Review","volume":"54","author":"Pang","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_15","unstructured":"Fort, S., Ren, J., and Lakshminarayanan, B. (2021, January 6\u201314). Exploring the limits of out-of-distribution detection. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2251","DOI":"10.1109\/TPAMI.2018.2857768","article-title":"Zero-shot learning\u2014A comprehensive evaluation of the good, the bad and the ugly","volume":"41","author":"Xian","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Xian, Y., Lorenz, T., Schiele, B., and Akata, Z. (2018, January 18\u201322). Feature generating networks for zero-shot learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00581"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Seo, S., and Kim, J. (2019). Hierarchical Semantic Loss and Confidence Estimator for Visual-Semantic Embedding-Based Zero-Shot Learning. Appl. Sci., 9.","DOI":"10.3390\/app9153133"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3614","DOI":"10.1109\/TPAMI.2020.2981604","article-title":"Recent advances in open set recognition: A survey","volume":"43","author":"Geng","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bertinetto, L., Mueller, R., Tertikas, K., Samangooei, S., and Lord, N.A. (2020, January 13\u201319). Making better mistakes: Leveraging class hierarchies with deep networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.","DOI":"10.1109\/CVPR42600.2020.01252"},{"key":"ref_21","unstructured":"Garnot, V.S.F., and Landrieu, L. (2021, January 22\u201325). Leveraging Class Hierarchies with Metric-Guided Prototype Learning. Proceedings of the British Machine Vision Conference (BMVC), Virtual."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Brust, C.A., Barz, B., and Denzler, J. (2021, January 10\u201315). Making every label count: Handling semantic imprecision by integrating domain knowledge. Proceedings of the International Conference on Pattern Recognition (ICPR), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9413283"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Brust, C.A., and Denzler, J. (2019, January 26\u201329). Integrating domain knowledge: Using hierarchies to improve deep classifiers. Proceedings of the Asian Conference on Pattern Recognition (ACPR), Auckland, New Zealand.","DOI":"10.1007\/978-3-030-41404-7_1"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Buzzelli, M., and Segantin, L. (2021). Revisiting the CompCars Dataset for Hierarchical Car Classification: New Annotations, Experiments, and Results. Sensors, 21.","DOI":"10.3390\/s21020596"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wu, T.Y., Morgado, P., Wang, P., Ho, C.H., and Vasconcelos, N. (2020, January 23\u201328). Solving long-tailed recognition with deep realistic taxonomic classifier. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.","DOI":"10.1007\/978-3-030-58598-3_11"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1016\/j.neucom.2021.07.008","article-title":"A knowledge-guide hierarchical learning method for long-tailed image classification","volume":"459","author":"Chen","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_27","unstructured":"Liu, W., Wen, Y., Yu, Z., and Yang, M. (2016, January 19\u201324). Large-margin softmax loss for convolutional neural networks. Proceedings of the International Conference on Machine Learning (ICML), New York, NY, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., and Song, L. (2017, January 21\u201326). Sphereface: Deep hypersphere embedding for face recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.713"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., and Liu, W. (2018, January 18\u201322). Cosface: Large margin cosine loss for deep face recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00552"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Deng, J., Guo, J., Xue, N., and Zafeiriou, S. (2019, January 16\u201320). Arcface: Additive angular margin loss for deep face recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00482"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhao, R., Qiao, Y., Wang, X., and Li, H. (2019, January 16\u201320). Adacos: Adaptively scaling cosine logits for effectively learning deep face representations. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01108"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhao, K., Xu, J., and Cheng, M.M. (2019, January 16\u201320). RegularFace: Deep Face Recognition via Exclusive Regularization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00123"},{"key":"ref_33","unstructured":"Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P. (2010). Caltech-UCSD Birds 200, Caltech. Technical Report CNS-TR-201."},{"key":"ref_34","unstructured":"Zhu, Z., Liang, D., Zhang, S., Huang, X., Li, B., and Hu, S. (July, January 26). Traffic-sign detection and classification in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA."},{"key":"ref_35","unstructured":"Ertler, C., Mislej, J., Ollmann, T., Porzi, L., and Kuang, Y. (2020, January 23\u201328). Traffic Sign Detection and Classification around the World. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK."},{"key":"ref_36","unstructured":"(2022, April 21). GitHub Repository. Available online: https:\/\/github.com\/idoiaruiz\/HierarchicalCosineLoss.git."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Deng, J., Berg, A.C., Li, K., and Li, F.-F. (2010, January 5\u201311). What does classifying more than 10,000 image categories tell us?. Proceedings of the European Conference on Computer Vision (ECCV), Hersonissos, Crete, Greece.","DOI":"10.1007\/978-3-642-15555-0_6"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4389\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:27:30Z","timestamp":1760138850000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4389"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,10]]},"references-count":37,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22124389"],"URL":"https:\/\/doi.org\/10.3390\/s22124389","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,6,10]]}}}