{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T17:17:05Z","timestamp":1763572625786,"version":"build-2065373602"},"reference-count":46,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,1,20]],"date-time":"2022-01-20T00:00:00Z","timestamp":1642636800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In some languages, Named Entity Recognition (NER) is severely hindered by complex linguistic structures, such as inflection, that will confuse the data-driven models when perceiving the word\u2019s actual meaning. This work tries to alleviate these problems by introducing a novel neural network based on morphological and syntactic grammars. The experiments were performed in four Nordic languages, which have many grammar rules. The model was named the NorG network (Nor: Nordic Languages, G: Grammar). In addition to learning from the text content, the NorG network also learns from the word writing form, the POS tag, and dependency. The proposed neural network consists of a bidirectional Long Short-Term Memory (Bi-LSTM) layer to capture word-level grammars, while a bidirectional Graph Attention (Bi-GAT) layer is used to capture sentence-level grammars. Experimental results from four languages show that the grammar-assisted network significantly improves the results against baselines. We also investigate how the NorG network works on each grammar component by some exploratory experiments.<\/jats:p>","DOI":"10.3390\/info13020049","type":"journal-article","created":{"date-parts":[[2022,1,20]],"date-time":"2022-01-20T22:51:06Z","timestamp":1642719066000},"page":"49","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Learning the Morphological and Syntactic Grammars for Named Entity Recognition"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9297-5985","authenticated-orcid":false,"given":"Mengtao","family":"Sun","sequence":"first","affiliation":[{"name":"Department of ICT and Natural Sciences, Norwegian University of Science and Technology, 6009 \u00c5lesund, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiang","family":"Yang","sequence":"additional","affiliation":[{"name":"China Telecom (Middle East) FZ-LLC., Dubai 500482, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9301-5989","authenticated-orcid":false,"given":"Hao","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Norwegian University of Science and Technology, 2815 Gj\u00f8vik, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2197-1538","authenticated-orcid":false,"given":"Mark","family":"Pasquine","sequence":"additional","affiliation":[{"name":"Department of International Business, Norwegian University of Science and Technology, 6009 \u00c5lesund, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1252-260X","authenticated-orcid":false,"given":"Ibrahim A.","family":"Hameed","sequence":"additional","affiliation":[{"name":"Department of ICT and Natural Sciences, Norwegian University of Science and Technology, 6009 \u00c5lesund, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Poll\u00e1k, F., Dor\u010d\u00e1k, P., and Markovi\u010d, P. (2021). Corporate Reputation of Family-Owned Businesses: Parent Companies vs. Their Brands. Information, 12.","DOI":"10.3390\/info12020089"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/TKDE.2020.2981314","article-title":"A survey on deep learning for named entity recognition","volume":"34","author":"Li","year":"2020","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3462441","article-title":"SymptomID: A framework for rapid symptom identification in pandemics using news reports","volume":"12","author":"Gu","year":"2021","journal-title":"ACM Trans. Manag. Inf. Syst. (TMIS)"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lee, D., Oh, B., Seo, S., and Lee, K.H. (2020, January 19\u201323). News recommendation with topic-enriched knowledge graphs. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual Conference.","DOI":"10.1145\/3340531.3411932"},{"key":"ref_5","unstructured":"Tanvir, H.M., Kittask, C., and Sirts, K. (June, January 31). EstBERT: A Pretrained Language-Specific BERT for Estonian. Proceedings of the 23rd Nordic Conference on Computational Linguistics, Reykjavik, Iceland."},{"key":"ref_6","unstructured":"Kutuzov, A., Barnes, J., Velldal, E., Ovrelid, L., and Oepen, S. (June, January 31). Large-Scale Contextualised Language Modelling for Norwegian. Proceedings of the 23rd Nordic Conference on Computational Linguistics, Reykjavik, Iceland."},{"key":"ref_7","unstructured":"Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv."},{"key":"ref_8","unstructured":"Shen, Y., Tan, S., Sordoni, A., and Courville, A. (2019, January 6\u20139). Ordered neurons: Integrating tree structures into recurrent neural networks. Proceedings of the ICLR, New Orleans, LA, USA."},{"key":"ref_9","unstructured":"Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019, January 3\u20135). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA."},{"key":"ref_10","unstructured":"Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit, O\u2019Reilly Media, Inc."},{"key":"ref_11","unstructured":"J\u00f8rgensen, F., Aasmoe, T., Husev\u00e5g, A.S.R., \u00d8vrelid, L., and Velldal, E. (2020, January 11\u201316). NorNE: Annotating named entities for Norwegian. Proceedings of the 12th Language Resources and Evaluation Conference, Le Palais du Pharo, France."},{"key":"ref_12","unstructured":"Hvingelby, R., Pauli, A.B., Barrett, M., Rosted, C., Lidegaard, L.M., and S\u00f8gaard, A. (2020, January 11\u201316). DaNE: A named entity resource for danish. Proceedings of the 12th Language Resources and Evaluation Conference, Le Palais du Pharo, France."},{"key":"ref_13","unstructured":"Derczynski, L. (2019). Simple natural language processing tools for Danish. arXiv."},{"key":"ref_14","unstructured":"Bick, E. (2004, January 26\u201328). A named entity recognizer for Danish. Proceedings of the Fourth International Conference on Language Resources and Evaluation, Lisbon, Portugal."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1093\/llc\/fqh045","article-title":"Named entity recognition for the mainland Scandinavian languages","volume":"20","author":"Johannessen","year":"2005","journal-title":"Lit. Linguist. Comput."},{"key":"ref_16","unstructured":"Luoma, J., Oinonen, M., Pyyk\u00f6nen, M., Laippala, V., and Pyysalo, S. (2020, January 11\u201316). A broad-coverage corpus for finnish named entity recognition. Proceedings of the 12th Language Resources and Evaluation Conference, Le Palais du Pharo, France."},{"key":"ref_17","unstructured":"Kettunen, K., and L\u00f6fberg, L. (2017, January 22\u201324). Tagging named entities in 19th century and modern Finnish newspaper material with a Finnish semantic tagger. Proceedings of the 21st Nordic Conference on Computational Linguistics, Gothenburg, Sweden."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1007\/s10579-019-09471-7","article-title":"A Finnish news corpus for named entity recognition","volume":"54","author":"Ruokolainen","year":"2020","journal-title":"Lang. Resour. Eval."},{"key":"ref_19","unstructured":"Akbik, A., Bergmann, T., Blythe, D., Rasul, K., Schweter, S., and Vollgraf, R. (2019, January 3\u20135). FLAIR: An easy-to-use framework for state-of-the-art NLP. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), Minneapolis, MN, USA."},{"key":"ref_20","unstructured":"Akbik, A., Blythe, D., and Vollgraf, R. (2018, January 20\u201326). Contextual string embeddings for sequence labeling. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, NM, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Akbik, A., Bergmann, T., and Vollgraf, R. (2019, January 3\u20135). Pooled contextualized embeddings for named entity recognition. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.","DOI":"10.18653\/v1\/N19-1078"},{"key":"ref_22","unstructured":"Lafferty, J., McCallum, A., and Pereira, F.C. (July, January 28). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the Eighteenth International Conference on Machine Learning, San Francisco, CA, USA."},{"key":"ref_23","unstructured":"Veli\u010dkovi\u0107, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2017). Graph attention networks. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Qi, P., and Manning, C.D. (November, January 31). Graph convolution over pruned dependency trees improves relation extraction. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1244"},{"key":"ref_25","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_26","unstructured":"Zhang, J., Shi, X., Xie, J., Ma, H., King, I., and Yeung, D.Y. (2018, January 7\u20139). Gaan: Gated attention networks for learning on large and spatiotemporal graphs. Proceedings of the Association for Uncertainty in Artificial Intelligence, Monterey, CA, USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lee, J.B., Rossi, R., and Kong, X. (2018, January 19\u201323). Graph classification using structural attention. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, New York, NY, USA.","DOI":"10.1145\/3219819.3219980"},{"key":"ref_28","unstructured":"Virtanen, A., Kanerva, J., Ilo, R., Luoma, J., Luotolahti, J., Salakoski, T., Ginter, F., and Pyysalo, S. (2019). Multilingual is not enough: BERT for Finnish. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"164717","DOI":"10.1109\/ACCESS.2020.3018688","article-title":"testing contextualized word embeddings to improve NER in Spanish clinical case narratives","volume":"8","author":"Akhtyamova","year":"2020","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yan, H., Jin, X., Meng, X., Guo, J., and Cheng, X. (2019, January 3\u20137). Event detection with multi-order graph convolution and aggregated attention. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1582"},{"key":"ref_31","unstructured":"Solberg, P.E., Skj\u00e6rholt, A., \u00d8vrelid, L., Hagen, K., and Johannessen, J.B. (2014, January 26\u201331). The Norwegian Dependency Treebank. Proceedings of the Ninth International Conference on Language Resources and Evaluation, Reykjavik, Iceland."},{"key":"ref_32","unstructured":"\u00d8vrelid, L., and Hohle, P. (2016, January 23\u201328). Universal Dependencies for Norwegian. Proceedings of the Tenth International Conference on Language Resources and Evaluation, Portoro\u017e, Slovenia."},{"key":"ref_33","unstructured":"Velldal, E., \u00d8vrelid, L., and Hohle, P. (2017, January 22\u201324). Joint UD parsing of Norwegian Bokm\u00e5l and Nynorsk. Proceedings of the 21st Nordic Conference of Computational Linguistics, Gothenburg, Sweden."},{"key":"ref_34","unstructured":"Buch-Kromann, M. (2003, January 14\u201315). The danish dependency treebank and the DTAG treebank tool. Proceedings of the 2nd Workshop on Treebanks and Linguistic Theories (TLT), V\u00e4xj\u00f6, Sweden."},{"key":"ref_35","unstructured":"Johannsen, A., Alonso, H.M., and Plank, B. (2015, January 11\u201312). Universal dependencies for danish. Proceedings of the International Workshop on Treebanks and Linguistic Theories (TLT14), Warsaw, Poland."},{"key":"ref_36","unstructured":"Keson, B. (2000). Vejledning til det Danske Morfosyntaktisk Taggede PAROLE-Korpus, Det Danske Sprog og Litteraturselskab (DSL). Technical Report."},{"key":"ref_37","unstructured":"Nivre, J., De Marneffe, M.C., Ginter, F., Goldberg, Y., Hajic, J., Manning, C.D., McDonald, R., Petrov, S., Pyysalo, S., and Silveira, N. (2016, January 23\u201328). Universal dependencies v1: A multilingual treebank collection. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC\u201916), Portoro\u017e, Slovenia."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1162\/tacl_a_00051","article-title":"Enriching word vectors with subword information","volume":"5","author":"Bojanowski","year":"2017","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yang, J., and Zhang, Y. (2018, January 15\u201320). NCRF++: An Open-source Neural Sequence Labeling Toolkit. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.","DOI":"10.18653\/v1\/P18-4013"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wolf, T., Chaumond, J., Debut, L., Sanh, V., Delangue, C., Moi, A., Cistac, P., Funtowicz, M., Davison, J., and Shleifer, S. (2020, January 16\u201320). Transformers: State-of-the-art natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Virtual.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"ref_41","unstructured":"Okazaki, N. (2021, May 20). CRFsuite: A Fast Implementation of Conditional Random Fields. Software Package. Available online: http:\/\/www.chokkan.org\/software\/crfsuite."},{"key":"ref_42","unstructured":"Kipf, T.N., and Welling, M. (2017, January 24\u201326). Semi-supervised classification with graph convolutional networks. Proceedings of the 2017 International Conference on Learning Representations, Toulon, France."},{"key":"ref_43","unstructured":"Huang, W., Zhang, T., Rong, Y., and Huang, J. (2018, January 2\u20138). Adaptive sampling towards fast graph representation learning. Proceedings of the 32nd Conference on Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_44","first-page":"3844","article-title":"Convolutional neural networks on graphs with fast localized spectral filtering","volume":"29","author":"Defferrard","year":"2016","journal-title":"Adv. Neural Inf. Processing Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Gui, T., Zou, Y., Zhang, Q., Peng, M., Fu, J., Wei, Z., and Huang, X.J. (2019, January 3\u20137). A lexicon-based graph neural network for chinese ner. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1096"},{"key":"ref_46","unstructured":"Honnibal, M., Montani, I., Van Landeghem, S., and Boyd, A. (2021, May 20). spaCy: Industrial-Strength Natural Language Processing in Python. Software Package. Available online: https:\/\/spacy.io."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/49\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:04:58Z","timestamp":1760133898000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/49"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,20]]},"references-count":46,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["info13020049"],"URL":"https:\/\/doi.org\/10.3390\/info13020049","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2022,1,20]]}}}