{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,29]],"date-time":"2025-11-29T07:37:38Z","timestamp":1764401858558,"version":"build-2065373602"},"reference-count":51,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2020,5,8]],"date-time":"2020-05-08T00:00:00Z","timestamp":1588896000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Traffic sign recognition is a classification problem that poses challenges for computer vision and machine learning algorithms. Although both computer vision and machine learning techniques have constantly been improved to solve this problem, the sudden rise in the number of unlabeled traffic signs has become even more challenging. Large data collation and labeling are tedious and expensive tasks that demand much time, expert knowledge, and fiscal resources to satisfy the hunger of deep neural networks. Aside from that, the problem of having unbalanced data also poses a greater challenge to computer vision and machine learning algorithms to achieve better performance. These problems raise the need to develop algorithms that can fully exploit a large amount of unlabeled data, use a small amount of labeled samples, and be robust to data imbalance to build an efficient and high-quality classifier. In this work, we propose a novel semi-supervised classification technique that is robust to small and unbalanced data. The framework integrates weakly-supervised learning and self-training with self-paced learning to generate attention maps to augment the training set and utilizes a novel pseudo-label generation and selection algorithm to generate and select pseudo-labeled samples. The method improves the performance by: (1) normalizing the class-wise confidence levels to prevent the model from ignoring hard-to-learn samples, thereby solving the imbalanced data problem; (2) jointly learning a model and optimizing pseudo-labels generated on unlabeled data; and (3) enlarging the training set to satisfy the hunger of deep learning models. Extensive evaluations on two public traffic sign recognition datasets demonstrate the effectiveness of the proposed technique and provide a potential solution for practical applications.<\/jats:p>","DOI":"10.3390\/s20092684","type":"journal-article","created":{"date-parts":[[2020,5,8]],"date-time":"2020-05-08T11:26:00Z","timestamp":1588937160000},"page":"2684","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Robust Semi-Supervised Traffic Sign Recognition via Self-Training and Weakly-Supervised Learning"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3072-6833","authenticated-orcid":false,"given":"Obed Tettey","family":"Nartey","sequence":"first","affiliation":[{"name":"Big Data Research Center, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guowu","family":"Yang","sequence":"additional","affiliation":[{"name":"Big Data Research Center, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"},{"name":"Guangxi Key Laboratory of Hybrid Computation and IC Design Analysis, Guangxi University for Nationalities, Nanning 530006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0992-4512","authenticated-orcid":false,"given":"Sarpong Kwadwo","family":"Asare","sequence":"additional","affiliation":[{"name":"School of Electronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinzhao","family":"Wu","sequence":"additional","affiliation":[{"name":"Guangxi Key Laboratory of Hybrid Computation and IC Design Analysis, Guangxi University for Nationalities, Nanning 530006, China"},{"name":"The School of Computer Science and Electronic Information, Guangxi University, Nanning 530004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lady Nadia","family":"Frempong","sequence":"additional","affiliation":[{"name":"The School of Computer Science and Electronic Information, Guangxi University, Nanning 530004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mana, N., Schwenker, F., and Trentin, E. (2012). Traffic Sign Classifier Adaption by Semi-supervised Co-training. Artificial Neural Networks in Pattern Recognition, Springer.","DOI":"10.1007\/978-3-642-33212-8"},{"key":"ref_2","unstructured":"Aghdam, H., and Heravi, E. (2017). Guide to Convolutional Neural Networks: A Practical Application to Traffic-Sign Detection and Classification, Springer International Publishing AG."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Houben, S., Stallkamp, J., Salmen, J., Schlipsing, M., and Igel, C. (2013, January 4\u20139). Detection of traffic signs in real-world images: The German traffic sign detection benchmark. Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA.","DOI":"10.1109\/IJCNN.2013.6706807"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Changzhen, X., Cong, W., Weixin, M., and Yanmei, S. (2016, January 13\u201315). A traffic sign detection algorithm based on deep convolutional neural network. Proceedings of the 2016 IEEE International Conference on Signal and Image Processing (ICSIP), Beijing, China.","DOI":"10.1109\/SIPROCESS.2016.7888348"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2022","DOI":"10.1109\/TITS.2015.2482461","article-title":"Towards Real-Time Traffic Sign Detection and Classification","volume":"17","author":"Yang","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"04076","DOI":"10.1051\/e3sconf\/201913604076","article-title":"Traffic marking recognition based on generating antagonistic neural network","volume":"136","author":"Xu","year":"2019","journal-title":"E3S Web Conf."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Sheikh, M.A.A., Kole, A., and Maity, T. (2016, January 21\u201323). Traffic sign detection and classification using colour feature and neural network. Proceedings of the 2016 International Conference on Intelligent Control Power and Instrumentation (ICICPI), Kolkata, India.","DOI":"10.1109\/ICICPI.2016.7859723"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1484","DOI":"10.1109\/TITS.2012.2209421","article-title":"Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey","volume":"13","author":"Mogelmose","year":"2012","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1016\/j.neucom.2014.11.026","article-title":"Traffic sign segmentation and classification using statistical learning methods","volume":"153","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1039","DOI":"10.1016\/j.patcog.2014.05.017","article-title":"Traffic Sign Detection via Interest Region Extraction","volume":"48","author":"Salti","year":"2015","journal-title":"Pattern Recogn."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1498","DOI":"10.1109\/TITS.2012.2208909","article-title":"Real-Time Detection and Recognition of Road Traffic Signs","volume":"13","author":"Greenhalgh","year":"2012","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.neucom.2014.12.111","article-title":"A novel traffic sign detection method via color segmentation and robust shape matching","volume":"169","author":"Li","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27\u201330). Learning Deep Features for Discriminative Localization. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.319"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ellahyani, A., Ansari, M.E., Jaafari, I.E., and Charfi, S. (2016). Traffic Sign Detection and Recognition using Features Combination and Random Forests. Int. J. Adv. Comput. Sci. Appl., 7.","DOI":"10.14569\/IJACSA.2016.070193"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1109\/TCYB.2016.2533424","article-title":"An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine","volume":"47","author":"Huang","year":"2017","journal-title":"IEEE Trans. Cybern."},{"key":"ref_16","unstructured":"Haloi, M. (2015). A novel pLSA based Traffic Signs Classification System. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.robot.2012.07.019","article-title":"Real-time traffic sign recognition in three stages","volume":"62","author":"Zaklouta","year":"2014","journal-title":"Robot. Auton. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Girshick, R.B., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1109\/TPAMI.2015.2389824","article-title":"Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition","volume":"37","author":"He","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_21","unstructured":"Shaham, U., Stanton, K., Li, H., Nadler, B., Basri, R., and Kluger, Y. (2018). SpectralNet: Spectral Clustering using Deep Neural Networks. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_23","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Durand, T., Mordan, T., Thome, N., and Cord, M. (2017, January 21\u201326). WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.631"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zou, Y., Yu, Z., Vijaya Kumar, B., and Wang, J. (2018). Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training. The European Conference on Computer Vision\u2014ECCV 2018, Springer International Publishing.","DOI":"10.1007\/978-3-030-01219-9_18"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Durand, T., Thome, N., and Cord, M. (2016, January 27\u201330). WELDON: Weakly Supervised Learning of Deep Convolutional Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.513"},{"key":"ref_27","unstructured":"Lon\u010dari\u0107, S., and Suba\u0161i\u0107, M. Convolutional Neural Networks for Croatian Traffic Signs Recognition. Proceedings of the Croatian Compter Vision Workshop, Year 2."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tabernik, D., and Skocaj, D. (2019). Deep Learning for Large-Scale Traffic-Sign Detection and Recognition. arXiv.","DOI":"10.1109\/TITS.2019.2913588"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1016\/j.neunet.2018.01.005","article-title":"Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods","volume":"99","year":"2018","journal-title":"Neural Netw."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"74602","DOI":"10.1109\/ACCESS.2019.2919125","article-title":"A Novel Method for Traffic Sign Recognition Based on DCGAN and MLP with PILAE Algorithm","volume":"7","author":"Mahmoud","year":"2019","journal-title":"IEEE Access"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sermanet, P., and LeCun, Y. (August, January 31). Traffic sign recognition with multi-scale Convolutional Networks. Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, CA, USA.","DOI":"10.1109\/IJCNN.2011.6033589"},{"key":"ref_32","unstructured":"Mariani, G., Scheidegger, F., Istrate, R., Bekas, C., and Malossi, A.C.I. (2018). BAGAN: Data Augmentation with Balancing GAN. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Diba, A., Sharma, V., Pazandeh, A., Pirsiavash, H., and Gool, L.V. (2017, January 21\u201326). Weakly Supervised Cascaded Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.545"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, X., You, S., Li, X., and Ma, H. (2018, January 18\u201322). Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00147"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ge, W., Yang, S., and Yu, Y. (2018, January 18\u201322). Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00139"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ge, Y., Zhang, R., Wang, X., Tang, X., and Luo, P. (2019, January 16\u201320). DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00548"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1561\/2200000013","article-title":"An Introduction to Conditional Random Fields","volume":"4","author":"Sutton","year":"2012","journal-title":"Found. Trends Mach. Learn."},{"key":"ref_39","unstructured":"Hu, T., and Qi, H. (2019). See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification. arXiv."},{"key":"ref_40","first-page":"1189","article-title":"Self-Paced Learning for Latent Variable Models","volume":"Volume 1","author":"Kumar","year":"2010","journal-title":"Proceedings of the 23rd International Conference on Neural Information Processing Systems"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Jiang, L., Meng, D., Zhao, Q., Shan, S., and Hauptmann, A.G. (2015, January 25\u201330). Self-Paced Curriculum Learning. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9608"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Dong, X., and Yang, Y. (2019, January 27\u201328). Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00087"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1641","DOI":"10.1109\/TPAMI.2018.2844853","article-title":"Few-Example Object Detection with Model Communication","volume":"41","author":"Dong","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2109","DOI":"10.1109\/ACCESS.2019.2962258","article-title":"Semi-Supervised Learning for Fine-Grained Classification with Self-Training","volume":"8","author":"Nartey","year":"2019","journal-title":"IEEE Access"},{"key":"ref_45","unstructured":"Ma, F., Meng, D., Xie, Q., Li, Z., and Dong, X. (2017, January 6\u201311). Self-Paced Co-training. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"He, Z., Nan, F., Li, X., Lee, S.J., and Yang, Y. (2019). Traffic Sign Recognition by Combining Global and Local Features Based on Semi-supervised Classification. IET Intell. Transport Syst.","DOI":"10.1049\/iet-its.2019.0409"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Mathias, M., Timofte, R., Benenson, R., and Van Gool, L. (2013, January 4\u20139). Traffic sign recognition\u2014How far are we from the solution?. Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA.","DOI":"10.1109\/IJCNN.2013.6707049"},{"key":"ref_48","unstructured":"Kingma, D., and Ba, J. (2014, January 14\u201316). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1991","DOI":"10.1109\/TITS.2014.2308281","article-title":"Traffic Sign Recognition With Hinge Loss Trained Convolutional Neural Networks","volume":"15","author":"Jin","year":"2014","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Saha, S., Amit Kamran, S., and Shihab Sabbir, A. (2018, January 21\u201323). Total Recall: Understanding Traffic Signs Using Deep Convolutional Neural Network. Proceedings of the 2018 21st International Conference of Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCITECHN.2018.8631925"},{"key":"ref_51","first-page":"11","article-title":"Improved VGG Model for Road Traffic Sign Recognition","volume":"57","author":"Zhou","year":"2018","journal-title":"Comput. Mater. Contin."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/9\/2684\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:26:48Z","timestamp":1760174808000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/9\/2684"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,8]]},"references-count":51,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2020,5]]}},"alternative-id":["s20092684"],"URL":"https:\/\/doi.org\/10.3390\/s20092684","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,5,8]]}}}