{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:29:07Z","timestamp":1760232547538,"version":"build-2065373602"},"reference-count":28,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2022,11,13]],"date-time":"2022-11-13T00:00:00Z","timestamp":1668297600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korea government (MSIT)","award":["2020-0-01373","2018R1A5A7059549","2020R1A2C1014037"],"award-info":[{"award-number":["2020-0-01373","2018R1A5A7059549","2020R1A2C1014037"]}]},{"DOI":"10.13039\/501100003725","name":"Korea government (*MSIT)","doi-asserted-by":"publisher","award":["2020-0-01373","2018R1A5A7059549","2020R1A2C1014037"],"award-info":[{"award-number":["2020-0-01373","2018R1A5A7059549","2020R1A2C1014037"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Semi-supervised learning is one of the active research topics these days. There is a trial that solves semi-supervised text classification with a generative adversarial network (GAN). However, its generator has a limitation in producing fake data distributions that are similar to real data distributions. Since the real data distribution is frequently changing, the generator could not create adequate fake data. To overcome this problem, we present a novel approach for semi-supervised learning for text classification based on generative adversarial networks, Linguistically Informed SeMi-Supervised GAN with Multiple Generators, LMGAN. LMGAN uses trained bidirectional encoder representations from transformers (BERT) and the discriminator from GAN-BERT. In addition, LMGAN has multiple generators and utilizes the hidden layers of BERT. To reduce the discrepancy between the distribution of fake data and real data distribution, LMGAN uses fine-tuned BERT and the discriminator from GAN-BERT. However, since injecting fine-tuned BERT could induce incorrect fake data distribution, we utilize linguistically meaningful intermediate hidden layer outputs of BERT to enrich fake data distribution. Our model shows well-distributed fake data compared to the earlier GAN-based approach that failed to generate adequate high-quality fake data. Moreover, we can get better performances with extremely limited amounts of labeled data, up to 20.0%, compared to the baseline GAN-based model.<\/jats:p>","DOI":"10.3390\/s22228761","type":"journal-article","created":{"date-parts":[[2022,11,14]],"date-time":"2022-11-14T04:30:52Z","timestamp":1668400252000},"page":"8761","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["LMGAN: Linguistically Informed Semi-Supervised GAN with Multiple Generators"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2069-4508","authenticated-orcid":false,"given":"Whanhee","family":"Cho","sequence":"first","affiliation":[{"name":"Department of Computer Science, Hanyang University, Seoul 04763, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9042-0599","authenticated-orcid":false,"given":"Yongsuk","family":"Choi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Hanyang University, Seoul 04763, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.knosys.2014.11.007","article-title":"Big data for Natural Language Processing: A streaming approach","volume":"79","author":"Agerri","year":"2015","journal-title":"Knowl.-Based Syst."},{"key":"ref_2","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_3","unstructured":"Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2022, August 23). Improving Language Understanding by Generative Pre-Training. Available online: https:\/\/openai.com\/blog\/language-unsupervised\/."},{"key":"ref_4","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv."},{"key":"ref_5","first-page":"6256","article-title":"Unsupervised Data Augmentation for Consistency Training","volume":"Volume 33","author":"Xie","year":"2020","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, J., Yang, Z., and Yang, D. (2020, January 5\u201310). MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.194"},{"key":"ref_7","unstructured":"Goodfellow, I., Shlens, J., and Szegedy, C. (2015, January 7\u20139). Explaining and Harnessing Adversarial Examples. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_8","unstructured":"Miyato, T., Dai, A.M., and Goodfellow, I. (2017, January 5\u20136). Adversarial Training Methods for Semi-Supervised Text Classification. Proceedings of the International Conference on Learning Representations, Singapore."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Park, J., Kim, G., and Kang, J. (2022, January 10\u201315). Consistency Training with Virtual Adversarial Discrete Perturbation. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, USA.","DOI":"10.18653\/v1\/2022.naacl-main.414"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Croce, D., Castellucci, G., and Basili, R. (2020, January 5\u201310). GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.191"},{"key":"ref_11","first-page":"2579","article-title":"Visualizing Data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_12","unstructured":"Odena, A. (2016). Semi-Supervised Learning with Generative Adversarial Networks. arXiv."},{"key":"ref_13","unstructured":"Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., and Chen, X. (2016, January 5\u201310). Improved Techniques for Training GANs. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_14","unstructured":"Liu, X., and Xiang, X. (2020). How Does GAN-based Semi-supervised Learning Work?. arXiv."},{"key":"ref_15","unstructured":"Dai, Z., Yang, Z., Yang, F., Cohen, W.W., and Salakhutdinov, R.R. (2017, January 4\u20139). Good semi-supervised learning that requires a bad gan. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_16","unstructured":"Donahue, J., Kr\u00e4henb\u00fchl, P., and Darrell, T. (2016). Adversarial feature learning. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Akcay, S., Atapour-Abarghouei, A., and Breckon, T.P. (2018, January 2\u20136). Ganomaly: Semi-supervised anomaly detection via adversarial training. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.","DOI":"10.1007\/978-3-030-20893-6_39"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_19","unstructured":"Jawahar, G., Sagot, B., and Seddah, D. (August, January 28). What Does BERT Learn about the Structure of Language?. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kovaleva, O., Romanov, A., Rogers, A., and Rumshisky, A. (2019, January 3\u20137). Revealing the Dark Secrets of BERT. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1445"},{"key":"ref_21","unstructured":"Kim, T., Choi, J., Edmiston, D., and Goo Lee, S. (May, January 26). Are Pre-trained Language Models Aware of Phrases? Simple but Strong Baselines for Grammar Induction. Proceedings of the International Conference on Learning Representations, Virtual."},{"key":"ref_22","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_23","unstructured":"Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C.A. (2019, January 8\u201314). MixMatch: A Holistic Approach to Semi-Supervised Learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1017\/S1351324905003955","article-title":"Learning question classifiers: The role of semantic information","volume":"12","author":"LI","year":"2006","journal-title":"Nat. Lang. Eng."},{"key":"ref_25","unstructured":"Prieditis, A., and Russell, S. (1995, January 9\u201312). NewsWeeder: Learning to Filter Netnews. Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, CA, USA."},{"key":"ref_26","unstructured":"Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A., and Potts, C. (2013, January 18\u201321). Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Williams, A., Nangia, N., and Bowman, S. (2018, January 1\u20136). A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1101"},{"key":"ref_28","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8761\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:17:13Z","timestamp":1760145433000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8761"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,13]]},"references-count":28,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22228761"],"URL":"https:\/\/doi.org\/10.3390\/s22228761","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,11,13]]}}}