{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T00:46:35Z","timestamp":1760402795364,"version":"build-2065373602"},"reference-count":69,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,12]],"date-time":"2020-01-12T00:00:00Z","timestamp":1578787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"South African Centre for Digital Language Resources (SADiLaR)","award":["n\/a"],"award-info":[{"award-number":["n\/a"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In this paper, the viability of neural network implementations of core technologies (the focus of this paper is on text technologies) for 10 resource-scarce South African languages is evaluated. Neural networks are increasingly being used in place of other machine learning methods for many natural language processing tasks with good results. However, in the South African context, where most languages are resource-scarce, very little research has been done on neural network implementations of core language technologies. In this paper, we address this gap by evaluating neural network implementations of four core technologies for ten South African languages. The technologies we address are part of speech tagging, named entity recognition, compound analysis and lemmatization. Neural architectures that performed well on similar tasks in other settings were implemented for each task and the performance was assessed in comparison with currently used machine learning implementations of each technology. The neural network models evaluated perform better than the baselines for compound analysis, are viable and comparable to the baseline on most languages for POS tagging and NER, and are viable, but not on par with the baseline, for Afrikaans lemmatization.<\/jats:p>","DOI":"10.3390\/info11010041","type":"journal-article","created":{"date-parts":[[2020,1,13]],"date-time":"2020-01-13T04:05:51Z","timestamp":1578888351000},"page":"41","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Viability of Neural Networks for Core Technologies for Resource-Scarce Languages"],"prefix":"10.3390","volume":"11","author":[{"given":"Melinda","family":"Loubser","sequence":"first","affiliation":[{"name":"Centre for Text Technology, North-West University, Potchefstroom 2520, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0493-3671","authenticated-orcid":false,"given":"Martin J.","family":"Puttkammer","sequence":"additional","affiliation":[{"name":"Centre for Text Technology, North-West University, Potchefstroom 2520, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,12]]},"reference":[{"key":"ref_1","first-page":"249","article-title":"Towards an 11 \u00d7 11 array for the degree of conjunctivism \/disjunctivism of the South African languages","volume":"11","author":"Prinsloo","year":"2002","journal-title":"Nord. J. Afr. Stud."},{"key":"ref_2","unstructured":"Eiselen, R., and Puttkammer, M.J. (2014, January 26\u201331). Developing Text Resources for Ten South African Languages. Proceedings of the Ninth International Conference on Language Resources and Evaluation, Reykjavik, Iceland."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"10052","DOI":"10.1073\/pnas.92.22.10052","article-title":"New trends in natural language processing: Statistical natural language processing","volume":"92","author":"Marcus","year":"1995","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Klein, E., and Veltman, F. (1991). The Trend towards Statistical Models in Natural Language Processing. Natural Language and Speech, Springer.","DOI":"10.1007\/978-3-642-77189-7"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"544","DOI":"10.1136\/amiajnl-2011-000464","article-title":"Natural language processing: An introduction","volume":"18","author":"Nadkarni","year":"2011","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.procs.2011.12.006","article-title":"The (Hopefully Near) Future of Language Technologies","volume":"7","year":"2011","journal-title":"Procedia Comput. Sci."},{"key":"ref_7","first-page":"2493","article-title":"Natural Language Processing (Almost) from Scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Young, T., Hazarika, D., Poria, S., and Cambria, E. (2017). Recent Trends in Deep Learning Based Natural Language Processing. arXiv.","DOI":"10.1109\/MCI.2018.2840738"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_10","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to Sequence Learning with Neural Networks. arXiv."},{"key":"ref_11","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2001, January 7\u201312). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics\u2014ACL \u201902, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1515\/pralin-2017-0013","article-title":"Is Neural Machine Translation the New State of the Art?","volume":"108","author":"Castilho","year":"2017","journal-title":"Prague Bull. Math. Linguist."},{"key":"ref_14","unstructured":"Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., and Macherey, K. (2016). Google\u2019s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv."},{"key":"ref_15","unstructured":"Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017). Convolutional Sequence to Sequence Learning. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bergmanis, T., and Goldwater, S. (2018, January 1\u20136). Context Sensitive Neural Lemmatisation with Lematus. Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1126"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kann, K., and Sch\u00fctze, H. (2016). Single-Model Encoder-Decoder with Explicit Morphological Representation for Reinflection. arXiv.","DOI":"10.18653\/v1\/P16-2090"},{"key":"ref_18","first-page":"1","article-title":"Neural Network Methods for Natural Language Processing","volume":"10","author":"Goldberg","year":"2017","journal-title":"Synth. Lect. Hum. Lang. Technol."},{"key":"ref_19","first-page":"431","article-title":"Educating Lia: The Development of a Linguistically Accurate Memory-Based Lemmatiser for Afrikaans","volume":"Volume 228","author":"Shi","year":"2007","journal-title":"Intelligent Information Processing III"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Mzamo, L., Helberg, A., and Bosch, S. (2015, January 26\u201327). Introducing XGL\u2014A lexicalised probabilistic graphical lemmatiser for isiXhosa. Proceedings of the 2015 PRASA-RobMech International Conference, Port Elizabeth, South Africa.","DOI":"10.1109\/RoboMech.2015.7359513"},{"key":"ref_21","unstructured":"van Huyssteen, G.B., and van Zaanen, M.M. (2004, January 1\u20132). Learning Compound Boundaries for Afrikaans Spelling Checking. Proceedings of the First Workshop on International Proofing Tools and Language Technologies, Patras, Greece."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"21","DOI":"10.4102\/lit.v29i1.99","article-title":"Die ontwikkeling van\u2019n woordafbreker en kompositumanaliseerder vir Afrikaans","volume":"29","author":"Pilon","year":"2008","journal-title":"Literator"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Schl\u00fcnz, G.I., Dlamini, N., and Kruger, R.P. (2016, January 8\u201312). Part-of-Speech Tagging and Chunking in Text-to-Speech Synthesis for South African Languages. Proceedings of the INTERSPEECH 2016, San Francisco, CA, USA.","DOI":"10.21437\/Interspeech.2016-1040"},{"key":"ref_24","unstructured":"Du Toit, J.S. (A Comparative Evaluation of Open-Source Part-of-Speech Taggers for South African Languages, 2017). A Comparative Evaluation of Open-Source Part-of-Speech Taggers for South African Languages, Unpublished Project Report."},{"key":"ref_25","unstructured":"Eiselen, R. (2016, January 23\u201328). Government Domain Named Entity Recognition for South African Languages. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC), Portoro\u017e, Slovenia."},{"key":"ref_26","unstructured":"Groenewald, H.J. (2006). Automatic Lemmatisation for Afrikaans. [Master\u2019s Thesis, North West University]."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Groenewald, H.J. (2009, January 31). Using technology transfer to advance automatic lemmatization for Setswana. Proceedings of the EACL 2009 Workshop on Language Technologies for African Languages, Athens, Greece.","DOI":"10.3115\/1564508.1564515"},{"key":"ref_28","first-page":"2","article-title":"Neurale netwerke as moontlike woordafkappingstegniek vir Afrikaans","volume":"22","author":"Fick","year":"2003","journal-title":"SA Tydskr. Nat. Tegnol."},{"key":"ref_29","unstructured":"Fick, M. (2013). \u2019n Masjienleerbenadering tot Woordafbreking in Afrikaans. [Ph.D. Thesis, UNISA]."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1093\/nsr\/nwx110","article-title":"Deep learning for natural language processing: Advantages and challenges","volume":"5","author":"Li","year":"2018","journal-title":"Natl. Sci. Rev."},{"key":"ref_31","unstructured":"(2018, August 14). Deep Learning Approaches for Low-Resource NLP, DeepLo. Available online: https:\/\/sites.google.com\/view\/deeplo18\/home."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hedderich, M.A., and Klakow, D. (2018). Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data. arXiv.","DOI":"10.18653\/v1\/W18-3402"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kann, K., Bjerva, J., Augenstein, I., Plank, B., and S\u00f8gaard, A. (2018, January 19). Character-level Supervision for Low-resource POS Tagging. Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP, Melbourne, Australia.","DOI":"10.18653\/v1\/W18-3401"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1613\/jair.4992","article-title":"A Primer on Neural Network Models for Natural Language Processing","volume":"57","author":"Goldberg","year":"2016","journal-title":"J. Artif. Intell. Res."},{"key":"ref_35","first-page":"1137","article-title":"A Neural Probabilistic Language Model","volume":"3","author":"Bengio","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_36","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv."},{"key":"ref_37","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality. arXiv."},{"key":"ref_38","unstructured":"Sutskever, I., Martens, J., and Hinton, G. (July, January 28). Generating Text with Recurrent Neural Networks. Proceedings of the 28th International Conference on Machine Learning, Bellevue, WA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. (2016). Enriching Word Vectors with Subword Information. arXiv.","DOI":"10.1162\/tacl_a_00051"},{"key":"ref_40","unstructured":"Nogueira dos Santos, C., and Zadrozny, B. (2014, January 21\u201326). Learning character-level representations for part-of-speech tagging. Proceedings of the 31st International Conference on Machine Learning, Beijing, China."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Pinter, Y., Guthrie, R., and Eisenstein, J. (2017). Mimicking Word Embeddings using Subword RNN\u2019s. arXiv.","DOI":"10.18653\/v1\/D17-1010"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Yu, X., Fale\u0144ska, A., and Vu, N.T. (2017). A General-Purpose Tagger with Convolutional Neural Networks. arXiv.","DOI":"10.18653\/v1\/W17-4118"},{"key":"ref_43","unstructured":"Wang, P., Qian, Y., Soong, F.K., He, L., and Zhao, H. (2015). Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural Architectures for Named Entity Recognition. arXiv.","DOI":"10.18653\/v1\/N16-1030"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Chiu, J.P.C., and Nichols, E. (2015). Named Entity Recognition with Bidirectional LSTM-CNN\u2019s. arXiv.","DOI":"10.1162\/tacl_a_00104"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Ma, X., and Hovy, E. (2016). End-to-end Sequence Labeling via Bi-directional LSTM-CNN\u2019s-CRF. arXiv.","DOI":"10.18653\/v1\/P16-1101"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Dos Santos, C.N., and Guimar\u00e3es, V. (2015). Boosting Named Entity Recognition with Neural Character Embeddings. arXiv.","DOI":"10.18653\/v1\/W15-3904"},{"key":"ref_48","unstructured":"Zennaki, O., Semmar, N., and Besacier, L. (November, January 30). Unsupervised and Lightly Supervised Part-of-Speech Tagging Using Recurrent Neural Networks. Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation (PACLIC), Shanghai, China."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Fang, M., and Cohn, T. (2017). Model Transfer for Tagging Low-resource Languages using a Bilingual Dictionary. arXiv.","DOI":"10.18653\/v1\/P17-2093"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Fang, M., and Cohn, T. (2016). Learning when to trust distant supervision: An application to low-resource POS tagging using cross-lingual projection. arXiv.","DOI":"10.18653\/v1\/K16-1018"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Plank, B., S\u00f8gaard, A., and Goldberg, Y. (2016). Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models; Auxiliary Loss. arXiv.","DOI":"10.18653\/v1\/P16-2067"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Faruqui, M., Tsvetkov, Y., Neubig, G., and Dyer, C. (2016, January 12\u201317). Morphological Inflection Generation Using Character Sequence to Sequence Learning. Proceedings of the NAACL-HLT 2016, San Diego, CA, USA.","DOI":"10.18653\/v1\/N16-1077"},{"key":"ref_53","unstructured":"Schnober, C., Eger, S., Dinh, E.-L.D., and Gurevych, I. (2016, January 11\u201316). Still not there? Comparing Traditional Sequence-to-Sequence Models to Encoder-Decoder Neural Networks on Monotone String Translation Tasks. Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan."},{"key":"ref_54","unstructured":"Hellwig, O. (2015, January 27\u201329). Using Recurrent Neural Networks for joint compound splitting and Sandhi resolution in Sanskrit. Proceedings of the 7th LTC, Pozna\u0144, Poland."},{"key":"ref_55","unstructured":"Dima, C., and Hinrichs, E. (2015, January 14\u201317). Automatic Noun Compound Interpretation using Deep Neural Networks and Word Embeddings. Proceedings of the 11th International Conference on Computational Semantics, London, UK."},{"key":"ref_56","first-page":"119","article-title":"\u2019n Woordsoortetiketteerder vir Afrikaans","volume":"26","author":"Pilon","year":"2008","journal-title":"S. Afr. Linguist. Appl. Lang. Stud."},{"key":"ref_57","first-page":"37","article-title":"A Comparison of Approaches to Word Class Tagging: Disjunctively vs. Conjunctively Written Bantu Languages","volume":"15","author":"Taljard","year":"2006","journal-title":"Nord. J. Afr. Stud."},{"key":"ref_58","unstructured":"Tjong, E.F., Sang, K., and De Meulder, F. (June, January 31). Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, Edmonton, AB, Canada."},{"key":"ref_59","unstructured":"(2018, August 02). CoNLL-U Format. Available online: http:\/\/universaldependencies.org\/format.html."},{"key":"ref_60","unstructured":"Grave, E., Bojanowski, P., Gupta, P., Joulin, A., and Mikolov, T. (2018). Learning Word Vectors for 157 Languages. arXiv."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Wiseman, S., and Rush, A.M. (2016, January 1\u20135). Sequence-to-Sequence Learning as Beam-Search Optimization. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA.","DOI":"10.18653\/v1\/D16-1137"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Gu, J., Lu, Z., Li, H., and Li, V.O.K. (2016, January 7\u201312). Incorporating Copying Mechanism in Sequence-to-Sequence Learning. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.","DOI":"10.18653\/v1\/P16-1154"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Klein, G., Kim, Y., Deng, Y., Senellart, J., and Rush, A.M. (2017). OpenNMT: Open-Source Toolkit for Neural Machine Translation. arXiv.","DOI":"10.18653\/v1\/P17-4012"},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Britz, D., Goldie, A., M.-Luong, T., and Le, Q. (2017). Massive Exploration of Neural Machine Translation Architectures. arXiv.","DOI":"10.18653\/v1\/D17-1151"},{"key":"ref_65","unstructured":"Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N. (2015). Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks. arXiv."},{"key":"ref_66","unstructured":"Bohnet, B., and Nivre, J. (2012, January 12\u201314). A Transition-Based System for Joint Part-of-Speech Tagging and Labeled Non-Projective Dependency Parsing. Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning Association for Computational Linguistics, Jeju Island, Korea."},{"key":"ref_67","unstructured":"University of Stuttgart (2018, August 16). Mate Tools. Available online: http:\/\/www.ims.uni-stuttgart.de\/forschung\/ressourcen\/werkzeuge\/matetools.en.html."},{"key":"ref_68","unstructured":"Dorogush, A.V., Ershov, V., and Gulin, A. (2017). CatBoost: Gradient boosting with categorical features support. arXiv."},{"key":"ref_69","unstructured":"Al-Rfou, R., Perozzi, B., and Skiena, S. (2013, January 8\u20139). Polyglot: Distributed Word Representations for Multilingual NLP. Proceedings of the Seventeenth Conference on Computational Natural Language Learning, Sofia, Bulgaria."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/1\/41\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:04:30Z","timestamp":1760364270000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/1\/41"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,12]]},"references-count":69,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["info11010041"],"URL":"https:\/\/doi.org\/10.3390\/info11010041","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2020,1,12]]}}}