{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T00:47:19Z","timestamp":1760402839109,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,4,1]],"date-time":"2021-04-01T00:00:00Z","timestamp":1617235200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Automatic recognition of visual objects using a deep learning approach has been successfully applied to multiple areas. However, deep learning techniques require a large amount of labeled data, which is usually expensive to obtain. An alternative is to use semi-supervised models, such as co-training, where multiple complementary views are combined using a small amount of labeled data. A simple way to associate views to visual objects is through the application of a degree of rotation or a type of filter. In this work, we propose a co-training model for visual object recognition using deep neural networks by adding layers of self-supervised neural networks as intermediate inputs to the views, where the views are diversified through the cross-entropy regularization of their outputs. Since the model merges the concepts of co-training and self-supervised learning by considering the differentiation of outputs, we called it Differential Self-Supervised Co-Training (DSSCo-Training). This paper presents some experiments using the DSSCo-Training model to well-known image datasets such as MNIST, CIFAR-100, and SVHN. The results indicate that the proposed model is competitive with the state-of-art models and shows an average relative improvement of 5% in accuracy for several datasets, despite its greater simplicity with respect to more recent approaches.<\/jats:p>","DOI":"10.3390\/e23040423","type":"journal-article","created":{"date-parts":[[2021,4,1]],"date-time":"2021-04-01T07:23:17Z","timestamp":1617261797000},"page":"423","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Co-Training for Visual Object Recognition Based on Self-Supervised Models Using a Cross-Entropy Regularization"],"prefix":"10.3390","volume":"23","author":[{"given":"Gabriel","family":"D\u00edaz","sequence":"first","affiliation":[{"name":"Departamento de Ciencias de Ingenier\u00eda, Facultad de Ingenier\u00eda, Universidad Andres Bello, Antonio Varas 880, 8370146 Santiago, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Billy","family":"Peralta","sequence":"additional","affiliation":[{"name":"Departamento de Ciencias de Ingenier\u00eda, Facultad de Ingenier\u00eda, Universidad Andres Bello, Antonio Varas 880, 8370146 Santiago, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luis","family":"Caro","sequence":"additional","affiliation":[{"name":"Departamento de Ingenier\u00eda Inform\u00e1tica, Facultad de Ingenier\u00eda, Universidad Cat\u00f3lica de Temuco, Rudecindo Ortega 2950, 4781312 Temuco, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8046-6983","authenticated-orcid":false,"given":"Orietta","family":"Nicolis","sequence":"additional","affiliation":[{"name":"Departamento de Ciencias de Ingenier\u00eda, Facultad de Ingenier\u00eda, Universidad Andres Bello, Antonio Varas 880, 8370146 Santiago, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,1]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"Visual object recognition","volume":"5","author":"Grauman","year":"2011","journal-title":"Synth. Lect. Artif. Intell. Mach. Learn."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s11263-019-01247-4","article-title":"Deep Learning for Generic Object Detection: A Survey","volume":"128","author":"Liu","year":"2019","journal-title":"Int. J. Comput. Vis."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Minciullo, L., Manhardt, F., Yoshikawa, K., Meier, S., Tombari, F., and Kobori, N. (2021, January 5\u20139). DB-GAN: Boosting Object Recognition Under Strong Lighting Conditions. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV2021), Virtual.","DOI":"10.1109\/WACV48630.2021.00298"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep Learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1038\/nmeth.4551","article-title":"Machine learning: Supervised methods","volume":"15","author":"Bzdok","year":"2018","journal-title":"Nat. Methods"},{"key":"ref_6","unstructured":"Dayan, P., Sahani, M., and Deback, G. (1999). Unsupervised learning. The MIT Encyclopedia of the Cognitive Sciences, The MIT Press."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"O\u2019Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G.V., Krpalkova, L., Riordan, D., and Walsh, J. (2019). Deep learning vs. traditional computer vision. Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, Springer.","DOI":"10.1007\/978-3-030-17795-9_10"},{"key":"ref_8","unstructured":"Han, B., Yao, Q., Yu, X., Niu, G., Xu, M., Hu, W., and Sugiyama, M. (2018, January 3\u20138). Co-teaching: Robust training of deep neural networks with extremely noisy labels. Proceedings of the Advances in Neural Information Processing Systems (NIPS 2018), Montr\u00e9al, QC, Canada."},{"key":"ref_9","first-page":"1","article-title":"Introduction to semi-supervised learning","volume":"3","author":"Zhu","year":"2009","journal-title":"Synth. Lect. Artif. Intell. Mach. Learn."},{"key":"ref_10","unstructured":"Blum, A., and Mitchell, T. Combining labeled and unlabeled data with co-training. Proceedings of the Eleventh Annual Conference on Computational Learning Theory (COLT\u2019 98)."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"124432","DOI":"10.1016\/j.jmaa.2020.124432","article-title":"Entropic regularization of continuous optimal transport problems","volume":"494","author":"Clason","year":"2021","journal-title":"J. Math. Anal. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Belousov, B., and Peters, J. (2019). Entropic regularization of markov decision processes. Entropy, 21.","DOI":"10.3390\/e21070674"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Peralta, B., Saavedra, A., Caro, L., and Soto, A. (2019). Mixture of experts with entropic regularization for data classification. Entropy, 21.","DOI":"10.3390\/e21020190"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Doersch, C., Gupta, A., and Efros, A.A. (2015, January 7\u201313). Unsupervised visual representation learning by context prediction. Proceedings of the IEEE International Conference on Computer Vision (ICCV 2015), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.167"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhai, X., Oliver, A., Kolesnikov, A., and Beyer, L. (November, January 27). S4l: Self-supervised semi-supervised learning. Proceedings of the IEEE International Conference on Computer Vision (ICCV2019), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00156"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., and Efros, A.A. (2016). Colorful image colorization. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46487-9_40"},{"key":"ref_17","unstructured":"Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv Preprint."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (2016, January 27\u201330). Context encoders: Feature learning by inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.278"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kim, D., Cho, D., Yoo, D., and Kweon, I.S. (2018, January 12\u201315). Learning Image Representations by Completing Damaged Jigsaw Puzzles. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00092"},{"key":"ref_20","unstructured":"Komodakis, N., and Gidaris, S. (May, January 30). Unsupervised representation learning by predicting image rotations. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Caron, M., Bojanowski, P., Joulin, A., and Douze, M. (2018, January 8\u201314). Deep clustering for unsupervised learning of visual features. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_9"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jing, L., and Tian, Y. (2020). Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey. IEEE Trans. Pattern Anal. Mach. Intell., 1.","DOI":"10.1109\/TPAMI.2020.2992393"},{"key":"ref_23","first-page":"1297","article-title":"Learning from crowds","volume":"11","author":"Raykar","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1979","DOI":"10.1109\/TPAMI.2018.2858821","article-title":"Virtual adversarial training: A regularization method for supervised and semi-supervised learning","volume":"41","author":"Miyato","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","unstructured":"Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. (2021, January 5\u20139). Improved techniques for training GANs. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV2021), Virtual."},{"key":"ref_26","unstructured":"Dai, Z., Yang, Z., Yang, F., Cohen, W., and Salakhutdinov, R. (2017, January 4\u20139). Good semisupervised learning that requires a bad GAN. Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wan, X. (2009, January 2\u20137). Co-training for cross-lingual sentiment classification. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Suntec, Singapore.","DOI":"10.3115\/1687878.1687913"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ardehaly, E.M., and Culotta, A. (2017, January 18\u201321). Co-training for demographic classification using deep learning from label proportions. Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW), New Orleans, LA, USA.","DOI":"10.1109\/ICDMW.2017.144"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Qiao, S., Shen, W., Zhang, Z., Wang, B., and Yuille, A. (2018, January 8\u201314). Deep co-training for semi-supervised image recognition. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01267-0_9"},{"key":"ref_30","unstructured":"Tarvainen, A., and Valpola, H. (2017, January 4\u20139). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kuznietsov, Y., Stuckler, J., and Leibe, B. (2017, January 21\u201326). Semi-supervised deep learning for monocular depth map prediction. Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition (CVPR 2017), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.238"},{"key":"ref_32","unstructured":"Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R.R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv Preprint."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., and Le, Q.V. (2019, January 16\u201320). Autoaugment: Learning augmentation strategies from data. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00020"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_36","unstructured":"Alex Krizhevsky, V.N., and Hinton, G. (2021, March 27). The CIFAR-10 Dataset. Available online: https:\/\/www.cs.toronto.edu\/~kriz\/cifar.html."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. Proc. AAAI Conf. Artif. Intell., 31, Available online: https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/11231.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_38","unstructured":"LeCun, Y., Cortes, C., and Burges, C.J. (2021, March 27). MNIST Dataset. Available online: http:\/\/yann.lecun.com\/exdb\/mnist\/."},{"key":"ref_39","unstructured":"Laine, S., and Aila, T. (2017, January 24\u201326). Temporal ensembling for semi-supervised learning. Proceedings of the 5th International Conference on Learning Representations, Toulon, France."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/4\/423\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:10:53Z","timestamp":1760364653000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/4\/423"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,1]]},"references-count":39,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["e23040423"],"URL":"https:\/\/doi.org\/10.3390\/e23040423","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2021,4,1]]}}}