{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T10:22:34Z","timestamp":1784802154638,"version":"3.55.0"},"reference-count":33,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,6,10]],"date-time":"2022-06-10T00:00:00Z","timestamp":1654819200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002855","name":"Research on Common Technology and Method Technology System of Post-marketing Clinical Research of Traditional Chinese Medicine","doi-asserted-by":"publisher","award":["2018YFC1707410"],"award-info":[{"award-number":["2018YFC1707410"]}],"id":[{"id":"10.13039\/501100002855","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002855","name":"Research on Common Technology and Method Technology System of Post-marketing Clinical Research of Traditional Chinese Medicine","doi-asserted-by":"publisher","award":["A201803082"],"award-info":[{"award-number":["A201803082"]}],"id":[{"id":"10.13039\/501100002855","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016083","name":"Hebei Province 333 Talent Funding Project \u201cBrain-like Intelligent Knowledge Discovery Technology Research\u201d","doi-asserted-by":"publisher","award":["2018YFC1707410"],"award-info":[{"award-number":["2018YFC1707410"]}],"id":[{"id":"10.13039\/100016083","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016083","name":"Hebei Province 333 Talent Funding Project \u201cBrain-like Intelligent Knowledge Discovery Technology Research\u201d","doi-asserted-by":"publisher","award":["A201803082"],"award-info":[{"award-number":["A201803082"]}],"id":[{"id":"10.13039\/100016083","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Text vectorization is the basic work of natural language processing tasks. High-quality vector representation with rich feature information can guarantee the quality of entity recognition and other downstream tasks in the field of traditional Chinese medicine (TCM). The existing word representation models mainly include the shallow models with relatively independent word vectors and the deep pre-training models with strong contextual correlation. Shallow models have simple structures but insufficient extraction of semantic and syntactic information, and deep pre-training models have strong feature extraction ability, but the models have complex structures and large parameter scales. In order to construct a lightweight word representation model with rich contextual semantic information, this paper enhances the shallow word representation model with weak contextual relevance at three levels: the part-of-speech (POS) of the predicted target words, the word order of the text, and the synonymy, antonymy and analogy semantics. In this study, we conducted several experiments in both intrinsic similarity analysis and extrinsic quantitative comparison. The results show that the proposed model achieves state-of-the-art performance compared to the baseline models. In the entity recognition task, the F1 value improved by 4.66% compared to the traditional continuous bag-of-words model (CBOW). The model is a lightweight word representation model, which reduces the training time by 51% compared to the pre-training language model BERT and reduces 89% in terms of memory usage.<\/jats:p>","DOI":"10.3390\/info13060296","type":"journal-article","created":{"date-parts":[[2022,6,10]],"date-time":"2022-06-10T10:25:12Z","timestamp":1654856712000},"page":"296","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Traditional Chinese Medicine Word Representation Model Augmented with Semantic and Grammatical Information"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3291-6227","authenticated-orcid":false,"given":"Yuekun","family":"Ma","sequence":"first","affiliation":[{"name":"College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, China"},{"name":"School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Hebei Provincial Key Laboratory of Industrial Intelligent Perception, Tangshan 063210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhongyan","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3456-5259","authenticated-orcid":false,"given":"Dezheng","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yechen","family":"Feng","sequence":"additional","affiliation":[{"name":"College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,10]]},"reference":[{"key":"ref_1","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv."},{"key":"ref_2","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J. (2013, January 5\u20138). Distributed representations of words and phrases and their compositionality. Proceedings of the 2014 Advances in Neural Information Processing Systems, Lake Tahoe, Nevada, USA."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C. (2014, January 25\u201329). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Peters, M., Neumann, M., Iyyer, M., and Gardner, M. (2018). Deep contextualized word representations. arXiv.","DOI":"10.18653\/v1\/N18-1202"},{"key":"ref_5","unstructured":"Vaswani, A., Shazeer, N., and Parmar, N. (2017, January 4\u20139). Attention is all you need. Proceedings of the 30th Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_6","unstructured":"Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lin, Y., Tan, Y., and Frank, R. (2019, January 1). Open Sesame: Getting inside BERT\u2019s Linguistic Knowledge. Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Florence, Italy.","DOI":"10.18653\/v1\/W19-4825"},{"key":"ref_8","unstructured":"Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R., Kim, N., and Das, B. (2019, January 6\u20139). What do you learn from context? Probing for sentence structure in contextualized word representations. Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1162\/tacl_a_00298","article-title":"What BERT is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models","volume":"8","author":"Ettinger","year":"2020","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e6439","DOI":"10.1002\/cpe.6439","article-title":"Grammar guided embedding based Chinese long text sentiment classification","volume":"33","author":"Zhang","year":"2021","journal-title":"Concurr. Comput. Pract. Exp."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chang, Y., Kong, L., Jia, K., and Meng, Q. (2021, January 29\u201331). Chinese named entity recognition method based on BERT. Proceedings of the 2021 IEEE International Conference on Data Science and Computer Application, Dalian, China.","DOI":"10.1109\/ICDSCA53499.2021.9650256"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sun, M., Yang, Q., Wang, H., Pasquine, M., and Hameed, I.A. (2022). Learning the Morphological and Syntactic Grammars for Named Entity Recognition. Information, 13.","DOI":"10.3390\/info13020049"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wu, C., Wu, F., Liu, J., Huang, Y., and Xie, X. (2019, January 3\u20137). Sentiment lexicon enhanced neural sentiment classification. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.","DOI":"10.1145\/3357384.3357973"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mao, X., Chang, S., Shi, J., Li, F., and Shi, R. (2019). Sentiment-aware word embedding for emotion classification. Appl. Sci., 9.","DOI":"10.3390\/app9071334"},{"key":"ref_15","unstructured":"Zhang, T., Wu, F., Katiyar, A., Weinberger, K.Q., and Artzi, Y. (2021, January 3\u20137). Revisiting Few-sample BERT Fine-tuning. Proceedings of the 9th International Conference on Learning Representations, Virtual Event, Austria."},{"key":"ref_16","unstructured":"Gao, Q. (2009). Fundamentals of Unified Linguistics, Science Press."},{"key":"ref_17","unstructured":"Fan, X. (1996). A Grammatical View of the Three Planes, Beijing Language and Culture University Press."},{"key":"ref_18","first-page":"1","article-title":"Research on verb\u2013core structure","volume":"1","author":"Fan","year":"2011","journal-title":"Bull. Linguist. Stud."},{"key":"ref_19","unstructured":"Jin, G. (2001). Semantic Computation Theory of Verbs in Modern Chinese, Peking University Press."},{"key":"ref_20","first-page":"73","article-title":"Study on semantic relations discovery based on key verbs in Chinese classical medical books","volume":"11","author":"Zhu","year":"2016","journal-title":"China Digit. Med."},{"key":"ref_21","first-page":"176","article-title":"Towards a Linguistic Theory of Specification Based on a Verb Grammar","volume":"10","author":"Babiniotis","year":"2022","journal-title":"Linguistics"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1017\/S1366728918001001","article-title":"Native and non-native (L1-Mandarin) speakers of English differ in online use of verb-based cues about sentence structure","volume":"22","author":"Qian","year":"2019","journal-title":"Biling. Lang. Cogn."},{"key":"ref_23","first-page":"274","article-title":"Construction research on opinion verbs-opinion targets intelligent computing","volume":"45","author":"Zhou","year":"2022","journal-title":"J. Shanxi Univ. (Nat. Sci. Ed.)"},{"key":"ref_24","unstructured":"Liu, Q., Ling, Z., Jiang, H., and Hu, Y. (2016). Part-of-Speech Relevance Weights for Learning Word Embeddings. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1108","DOI":"10.1016\/j.neucom.2015.07.046","article-title":"A novel word embedding learning model using the dissociation between nouns and verbs","volume":"171","author":"Hu","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_26","first-page":"1976","article-title":"The improved model for word2vec based on part of speech and word order","volume":"46","author":"Pan","year":"2018","journal-title":"Acta Electonica Sin."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wang, Z., Liu, X., Wang, L., Qiao, Y., Xie, X., and Fowlkes, C. (2018, January 12\u201315). Structured triplet learning with pos-tag guided attention for visual question answering. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00209"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1049\/trit.2020.0055","article-title":"Improving word vector model with part-of-speech and dependency grammar information","volume":"5","author":"Deng","year":"2020","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ren, X., Zhang, L., Ye, W., Hua, H., and Zhang, S. (2018, January 4\u20137). Attention enhanced Chinese word embeddings. Proceedings of the 27th International Conference on Artificial Neural Networks, Rhodes, Greece.","DOI":"10.1007\/978-3-030-01418-6_16"},{"key":"ref_30","first-page":"80","article-title":"The impact of Inner Canon\u2019s language feature on its translation","volume":"14","author":"Yang","year":"2015","journal-title":"J. Zhejiang Bus. Technol. Inst."},{"key":"ref_31","unstructured":"Qian, C. (1990). Neijing Language Research, People\u2019s Medical Publishing House."},{"key":"ref_32","unstructured":"Wang, H. (2017). Study on the Thought of Numerology in Huangdi Neijing. [Ph.D. Thesis, Beijing University of Chinese Medicine]."},{"key":"ref_33","first-page":"4899","article-title":"Discussion on the classification according to manifestation is the essence of Chinese medicine theory","volume":"31","author":"Zhang","year":"2016","journal-title":"China J. Tradit. Chin. Med. Pharm."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/6\/296\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:27:31Z","timestamp":1760138851000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/6\/296"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,10]]},"references-count":33,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["info13060296"],"URL":"https:\/\/doi.org\/10.3390\/info13060296","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,10]]}}}