{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T10:20:08Z","timestamp":1777890008906,"version":"3.51.4"},"reference-count":29,"publisher":"SAGE Publications","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["WEB"],"published-print":{"date-parts":[[2020,9,30]]},"abstract":"<jats:p>A parsimonious convolutional neural network (CNN) for text document classification that replicates the ease of use and high classification performance of linear methods is presented. This new CNN architecture can leverage locally trained latent semantic analysis (LSA) word vectors. The architecture is based on parallel 1D convolutional layers with small window sizes, ranging from 1 to 5 words. To test the efficacy of the new CNN architecture, three balanced text datasets that are known to perform exceedingly well with linear classifiers were evaluated. Also, three additional imbalanced datasets were evaluated to gauge the robustness of the LSA vectors and small window sizes. The new CNN architecture consisting of 1 to 4-grams, coupled with LSA word vectors, exceeded the accuracy of all linear classifiers on balanced datasets with an average improvement of 0.73%. In four out of the total six datasets, the LSA word vectors provided a maximum classification performance on par with or better than word2vec vectors in CNNs. Furthermore, in four out of the six datasets, the new CNN architecture provided the highest classification performance. Thus, the new CNN architecture and LSA word vectors could be used as a baseline method for text classification tasks.<\/jats:p>","DOI":"10.3233\/web-200445","type":"journal-article","created":{"date-parts":[[2020,8,11]],"date-time":"2020-08-11T17:24:10Z","timestamp":1597166650000},"page":"239-248","source":"Crossref","is-referenced-by-count":2,"title":["Document classification using convolutional neural networks with small window sizes and latent semantic analysis"],"prefix":"10.1177","volume":"18","author":[{"given":"Eren","family":"Gultepe","sequence":"first","affiliation":[{"name":"Department of Computer Science, Southern Illinois University Edwardsville, Edwardsville, IL 62026, USA. E-mail:\u00a0egultep@siue.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mehran","family":"Kamkarhaghighi","sequence":"additional","affiliation":[{"name":"Department of Electrical, Computer, and Software Engineering, Ontario Tech University, Oshawa, ON L1G 0C5, Canada. E-mails:\u00a0mehran.kamkarhaghighi@uoit.net,\u00a0masoud.makrehchi@uoit.ca"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masoud","family":"Makrehchi","sequence":"additional","affiliation":[{"name":"Department of Electrical, Computer, and Software Engineering, Ontario Tech University, Oshawa, ON L1G 0C5, Canada. E-mails:\u00a0mehran.kamkarhaghighi@uoit.net,\u00a0masoud.makrehchi@uoit.ca"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"issue":"8","key":"10.3233\/WEB-200445_ref2","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: A review and new perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"10.3233\/WEB-200445_ref3","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"10.3233\/WEB-200445_ref4","doi-asserted-by":"crossref","unstructured":"A.\u00a0Conneau, H.\u00a0Schwenk, Y.\u00a0LeCun and L.\u00a0Barrault, Very deep convolutional networks for text classification, in: Long Papers\u00a0\u2013 Continued, Vol. 1, Association for Computational Linguistics (ACL), 2017, pp.\u00a01107\u20131116.","DOI":"10.18653\/v1\/E17-1104"},{"key":"10.3233\/WEB-200445_ref5","doi-asserted-by":"crossref","unstructured":"T.\u00a0Davidson, D.\u00a0Warmsley, M.\u00a0Macy and I.\u00a0Weber, Automated hate speech detection and the problem of offensive language, in: Eleventh International AAAI Conference on Web and Social Media, 2017.","DOI":"10.1609\/icwsm.v11i1.14955"},{"key":"10.3233\/WEB-200445_ref6","first-page":"1871","article-title":"LIBLINEAR: A library for large linear classification","volume":"9","author":"Fan","year":"2008","journal-title":"Journal of Machine Learning Research"},{"issue":"5","key":"10.3233\/WEB-200445_ref7","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","article-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures","volume":"18","author":"Graves","year":"2005","journal-title":"Neural Networks"},{"issue":"8","key":"10.3233\/WEB-200445_ref10","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"10.3233\/WEB-200445_ref11","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1016\/j.neucom.2015.01.031","article-title":"A two-stage Markov blanket based feature selection algorithm for text classification","volume":"157","author":"Javed","year":"2015","journal-title":"Neurocomputing"},{"key":"10.3233\/WEB-200445_ref12","doi-asserted-by":"crossref","unstructured":"Y.\u00a0Jia, E.\u00a0Shelhamer, J.\u00a0Donahue, S.\u00a0Karayev, J.\u00a0Long, R.\u00a0Girshick, S.\u00a0Guadarrama and T.\u00a0Darrell, Caffe: Convolutional architecture for fast feature embedding, in: Proceedings of the 22nd ACM International Conference on Multimedia, ACM, 2014, pp.\u00a0675\u2013678.","DOI":"10.1145\/2647868.2654889"},{"key":"10.3233\/WEB-200445_ref13","doi-asserted-by":"crossref","unstructured":"R.\u00a0Johnson and T.\u00a0Zhang, Effective use of word order for text categorization with convolutional neural networks, in: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Denver, Colorado, 2015, pp.\u00a0103\u2013112.","DOI":"10.3115\/v1\/N15-1011"},{"key":"10.3233\/WEB-200445_ref14","unstructured":"R.\u00a0Johnson and T.\u00a0Zhang, Semi-supervised convolutional neural networks for text categorization via region embedding, in: Advances in Neural Information Processing Systems 28, C.\u00a0Cortes, N.D.\u00a0Lawrence, D.D.\u00a0Lee, M.\u00a0Sugiyama and R.\u00a0Garnett, eds, Curran Associates, Inc., 2015, pp.\u00a0919\u2013927."},{"issue":"1","key":"10.3233\/WEB-200445_ref15","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1108\/eb026526","article-title":"A statistical interpretation of term specificity and its application in retrieval","volume":"28","author":"Jones","year":"1972","journal-title":"Journal of Documentation"},{"key":"10.3233\/WEB-200445_ref17","unstructured":"H.\u00a0Kim, P.\u00a0Howland and H.\u00a0Park, Dimension reduction in text classification with support vector machines, in: Journal of Machine Learning Research, 2005, pp.\u00a037\u201353."},{"key":"10.3233\/WEB-200445_ref18","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1181"},{"key":"10.3233\/WEB-200445_ref19","unstructured":"T.K.\u00a0Landauer, D.S.\u00a0McNamara, S.\u00a0Dennis and W.\u00a0Kintsch, Handbook of Latent Semantic Analysis, Psychology Press, 2013."},{"issue":"2","key":"10.3233\/WEB-200445_ref21","doi-asserted-by":"publisher","first-page":"167","DOI":"10.3233\/SW-140134","article-title":"DBpedia\u00a0\u2013 A large-scale, multilingual knowledge base extracted from Wikipedia","volume":"6","author":"Lehmann","year":"2015","journal-title":"Semantic Web"},{"key":"10.3233\/WEB-200445_ref22","doi-asserted-by":"crossref","unstructured":"J.\u00a0Leskovec, A.\u00a0Rajaraman and J.D.\u00a0Ullman, Mining of Massive Datasets, Cambridge University Press, 2014.","DOI":"10.1017\/CBO9781139924801"},{"key":"10.3233\/WEB-200445_ref23","unstructured":"O.\u00a0Levy and Y.\u00a0Goldberg, Neural word embedding as implicit matrix factorization, in: Advances in Neural Information Processing Systems 27, Z.\u00a0Ghahramani, M.\u00a0Welling, C.\u00a0Cortes, N.D.\u00a0Lawrence and K.Q.\u00a0Weinberger, eds, Curran Associates, Inc., 2014, pp.\u00a02177\u20132185."},{"key":"10.3233\/WEB-200445_ref24","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1162\/tacl_a_00134","article-title":"Improving distributional similarity with lessons learned from word embeddings","volume":"3","author":"Levy","year":"2015","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"10.3233\/WEB-200445_ref26","unstructured":"A.L.\u00a0Maas, R.E.\u00a0Daly, P.T.\u00a0Pham, D.\u00a0Huang, A.Y.\u00a0Ng and C.\u00a0Potts, Learning word vectors for sentiment analysis, in: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Portland, Oregon, USA, 2011, pp.\u00a0142\u2013150."},{"key":"10.3233\/WEB-200445_ref27","unstructured":"T.\u00a0Mikolov, I.\u00a0Sutskever, K.\u00a0Chen, G.S.\u00a0Corrado and J.\u00a0Dean, Distributed representations of words and phrases and their compositionality, in: Advances in Neural Information Processing Systems, 2013, pp.\u00a03111\u20133119."},{"key":"10.3233\/WEB-200445_ref28","unstructured":"R.\u00a0Pascanu, T.\u00a0Mikolov and Y.\u00a0Bengio, On the difficulty of training recurrent neural networks, in: Proceedings of the 30th International Conference on International Conference on Machine Learning, Vol.\u00a028, JMLR.org, 2013, pp. III\u20131310\u2013III-1318. http:\/\/dl.acm.org\/citation.cfm?id=3042817.3043083."},{"key":"10.3233\/WEB-200445_ref29","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2014.131"},{"issue":"3","key":"10.3233\/WEB-200445_ref30","first-page":"287","article-title":"Distributed text classification with an ensemble kernel-based learning approach, systems, man, and cybernetics, part C: Applications and reviews","volume":"40","author":"Silva","year":"2010","journal-title":"IEEE Transactions on"},{"issue":"1","key":"10.3233\/WEB-200445_ref31","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"The Journal of Machine Learning Research"},{"key":"10.3233\/WEB-200445_ref32","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1016\/j.neucom.2015.09.096","article-title":"Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification","volume":"174","author":"Wang","year":"2016","journal-title":"Neurocomputing"},{"key":"10.3233\/WEB-200445_ref33","unstructured":"S.\u00a0Wang and C.D.\u00a0Manning, Baselines and bigrams: Simple, good sentiment and topic classification, in: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Volume 2, Association for Computational Linguistics, 2012, pp.\u00a090\u201394."},{"key":"10.3233\/WEB-200445_ref34","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-014-0430-y"},{"key":"10.3233\/WEB-200445_ref36","unstructured":"X.\u00a0Zhang, J.\u00a0Zhao and Y.\u00a0LeCun, Character-level convolutional networks for text classification, in: Advances in Neural Information Processing Systems, 2015, pp.\u00a0649\u2013657."}],"container-title":["Web Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/WEB-200445","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T05:27:21Z","timestamp":1777613241000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/WEB-200445"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,30]]},"references-count":29,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/web-200445","relation":{},"ISSN":["2405-6464","2405-6456"],"issn-type":[{"value":"2405-6464","type":"electronic"},{"value":"2405-6456","type":"print"}],"subject":[],"published":{"date-parts":[[2020,9,30]]}}}