{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T15:15:03Z","timestamp":1777130103408,"version":"3.51.4"},"reference-count":56,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,2,22]],"date-time":"2020-02-22T00:00:00Z","timestamp":1582329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Increasingly, popular online museums have significantly changed the way people acquire cultural knowledge. These online museums have been generating abundant amounts of cultural relics data. In recent years, researchers have used deep learning models that can automatically extract complex features and have rich representation capabilities to implement named-entity recognition (NER). However, the lack of labeled data in the field of cultural relics makes it difficult for deep learning models that rely on labeled data to achieve excellent performance. To address this problem, this paper proposes a semi-supervised deep learning model named SCRNER (Semi-supervised model for Cultural Relics\u2019 Named Entity Recognition) that utilizes the bidirectional long short-term memory (BiLSTM) and conditional random fields (CRF) model trained by seldom labeled data and abundant unlabeled data to attain an effective performance. To satisfy the semi-supervised sample selection, we propose a repeat-labeled (relabeled) strategy to select samples of high confidence to enlarge the training set iteratively. In addition, we use embeddings from language model (ELMo) representations to dynamically acquire word representations as the input of the model to solve the problem of the blurred boundaries of cultural objects and Chinese characteristics of texts in the field of cultural relics. Experimental results demonstrate that our proposed model, trained on limited labeled data, achieves an effective performance in the task of named entity recognition of cultural relics.<\/jats:p>","DOI":"10.3390\/e22020252","type":"journal-article","created":{"date-parts":[[2020,2,24]],"date-time":"2020-02-24T03:33:43Z","timestamp":1582515223000},"page":"252","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Semi-Supervised Bidirectional Long Short-Term Memory and Conditional Random Fields Model for Named-Entity Recognition Using Embeddings from Language Models Representations"],"prefix":"10.3390","volume":"22","author":[{"given":"Min","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, Northwest University, Xi\u2019an 710127, China"},{"name":"School of Engineering and Technology, Xi\u2019an Fanyi University, 710105 Xi\u2019an, China"}]},{"given":"Guohua","family":"Geng","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Northwest University, Xi\u2019an 710127, China"}]},{"given":"Jing","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Northwest University, Xi\u2019an 710127, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.websem.2008.08.001","article-title":"Semantic annotation and search of cultural-heritage collections: The MultimediaN E-Culture demonstrator","volume":"6","author":"Schreiber","year":"2008","journal-title":"J. Web Semant."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Brando, C., Frontini, F., and Ganascia, J.G. (2015, January 8\u201311). Disambiguation of named entities in cultural heritage texts using linked data sets. Proceedings of the East European Conference on Advances in Databases and Information Systems, Poitiers, France.","DOI":"10.1007\/978-3-319-23201-0_51"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ardissono, L., Lucenteforte, M., Mauro, N., Savoca, A., Voghera, A., and La Riccia, L. (2016, January 6\u20139). Exploration of cultural heritage information via textual search queries. Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, Florence, Italy.","DOI":"10.1145\/2957265.2962648"},{"key":"ref_4","unstructured":"Hyv\u00f6nen, E., and Rantala, H. (2019). Knowledge-based Relation Discovery in Cultural Heritage Knowledge Graphs, CEUR-WS."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"White, M., Patoli, Z., and Pascu, T. (November, January 28). Knowledge networking through social media for a digital heritage resource. Proceedings of the 2013 Digital Heritage International Congress (DigitalHeritage), Marseille, France.","DOI":"10.1109\/DigitalHeritage.2013.6744787"},{"key":"ref_6","unstructured":"Yadav, V., and Bethard, S. (2019). A survey on recent advances in named entity recognition from deep learning models. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Peng, N., and Dredze, M. (2015, January 17\u201321). Named entity recognition for Chinese social media with jointly trained embeddings. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1064"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018, January 1\u20136). Deep Contextualized Word Representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1202"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3040","DOI":"10.1109\/TPDS.2014.2368568","article-title":"Hadoop recognition of biomedical named entity using conditional random fields","volume":"26","author":"Li","year":"2014","journal-title":"IEEE Trans. Parallel Distrib."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"905","DOI":"10.1016\/j.jbi.2008.12.012","article-title":"Feature selection techniques for maximum entropy based biomedical named entity recognition","volume":"42","author":"Saha","year":"2009","journal-title":"J. Biomed. Inf."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yang, H., and Gao, H. (2018). Toward sustainable virtualized healthcare: Extracting medical entities from Chinese online health consultations using deep neural networks. Sustainability, 10.","DOI":"10.3390\/su10093292"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1109\/TNN.2009.2015974","article-title":"Semi-supervised learning (chapelle, o. et al., eds.; 2006) [book reviews]","volume":"20","author":"Chapelle","year":"2009","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1007\/s10115-013-0706-y","article-title":"Self-labeled techniques for semi-supervised learning: Taxonomy, software and empirical study","volume":"42","author":"Triguero","year":"2015","journal-title":"Knowled. Inf. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Vesel\u00fd, K., Hannemann, M., and Burget, L. (2013, January 8\u201312). Semi-supervised training of deep neural networks. Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic.","DOI":"10.1109\/ASRU.2013.6707741"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Livieris, I.E., Drakopoulou, K., Mikropoulos, T.A., Tampakas, V., and Pintelas, P. (2018). An ensemble-based semi-supervised approach for predicting students\u2019 performance. Research on e-Learning and ICT in Education, Springer.","DOI":"10.1007\/978-3-319-95059-4_2"},{"key":"ref_16","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_17","first-page":"1326","article-title":"A study of neural word embeddings for named entity recognition in clinical text","volume":"2015","author":"Wu","year":"2015","journal-title":"AMIA"},{"key":"ref_18","first-page":"2741","article-title":"Character-Aware neural language models","volume":"Volume 3","author":"Kim","year":"2016","journal-title":"Proceedings of the 30th AAAI Conference on Artificial Intelligence"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Dong, C., Zhang, J., Zong, C., Hattori, M., and Di, H. (2016). Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. Natural Language Understanding and Intelligent Applications, Springer.","DOI":"10.1007\/978-3-319-50496-4_20"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xu, C., Wang, F., Han, J., and Li, C. (2019, January 3\u20137). Exploiting Multiple Embeddings for Chinese Named Entity Recognition. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.","DOI":"10.1145\/3357384.3358117"},{"key":"ref_21","unstructured":"Chen, X., Xu, L., Liu, Z., Sun, M., and Luan, H. (August, January 25). Joint learning of character and word embeddings. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zeng, D., Sun, C., Lin, L., and Liu, B. (2017). LSTM-CRF for drug-named entity recognition. Entropy, 19.","DOI":"10.3390\/e19060283"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yang, J., Liu, Y., Qian, M., Guan, C., and Yuan, X. (2019). Information Extraction from Electronic Medical Records Using Multitask Recurrent Neural Network with Contextual Word Embedding. Appl. Sci., 9.","DOI":"10.3390\/app9183658"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Strakov\u00e1, J., Straka, M., and Haji\u010d, J. (2019). Neural architectures for nested NER through linearization. arXiv.","DOI":"10.18653\/v1\/P19-1527"},{"key":"ref_25","unstructured":"Dogan, C., Dutra, A., Gara, A., Gemma, A., Shi, L., Sigamani, M., and Walters, E. (2019). Fine-Grained Named Entity Recognition using ELMo and Wikidata. arXiv."},{"key":"ref_26","unstructured":"Isozaki, H., and Kazawa, H. (September, January 24). Efficient support vector classifiers for named entity recognition. Proceedings of the 19th International Conference on Computational Linguistics-Volume 1. Association for Computational Linguistics, Taipei, Taiwan."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bender, O., Och, F.J., and Ney, H. (June, January 31). Maximum entropy models for named entity recognition. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4, Association for Computational Linguistics, Edmonton, AB, Canada.","DOI":"10.3115\/1119176.1119196"},{"key":"ref_28","unstructured":"Chen, W., Zhang, Y., and Isahara, H. (2020, February 22). Chinese Named Entity Recognition with Conditional Random Fields. Available online: https:\/\/www.aclweb.org\/anthology\/W06-0100."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"143","DOI":"10.5120\/72-166","article-title":"Conditional random field based named entity recognition in geological text","volume":"1","author":"Sobhana","year":"2010","journal-title":"IJCA"},{"key":"ref_30","unstructured":"Limsopatham, N., and Collier, N. (2016, January 11). Bidirectional LSTM for Named Entity Recognition in Twitter Messages. Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT), Osaka, Japan."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hammerton, J. (June, January 31). Named entity recognition with long short-term memory. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003-Volume 4, Association for Computational Linguistics, Edmonton, AB, Canada.","DOI":"10.3115\/1119176.1119202"},{"key":"ref_32","unstructured":"Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Xu, K., Zhou, Z., Hao, T., and Liu, W. (2017, January 9\u201311). A bidirectional LSTM and conditional random fields approach to medical named entity recognition. Proceedings of the International Conference on Advanced Intelligent Systems and Informatics, Cairo, Egypt.","DOI":"10.1007\/978-3-319-64861-3_33"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural architectures for named entity recognition. arXiv.","DOI":"10.18653\/v1\/N16-1030"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ji, H., and Grishman, R. (2006, January 22). Data selection in semi-supervised learning for name tagging. Proceedings of the Workshop on Information Extraction Beyond the Document, Association for Computational Linguistics, Sydney, Australia.","DOI":"10.3115\/1641408.1641414"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2142","DOI":"10.1109\/TASLP.2018.2856625","article-title":"Cross-domain and semisupervised named entity recognition in Chinese social media: A unified model","volume":"26","author":"Xu","year":"2018","journal-title":"IEEE-ACM Trans. Audio Speech Lang. Process."},{"key":"ref_37","unstructured":"Liao, W., and Veeramachaneni, S. (2020, February 22). A Simple Semi-Supervised Algorithm for Named Entity Recognition. Available online: https:\/\/www.aclweb.org\/anthology\/W09-2208."},{"key":"ref_38","unstructured":"Liu, X., Zhang, S., Wei, F., and Zhou, M. (2011, January 19\u201324). Recognizing named entities in tweets. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, Association for Computational Linguistics, Portland, Oregon."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Luan, Y., Ostendorf, M., and Hajishirzi, H. (2017, January 7\u201311). Scientific Information Extraction with Semi-supervised Neural Tagging. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1279"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neur. Comput."},{"key":"ref_41","first-page":"3771","article-title":"Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding","volume":"8","author":"Mesnil","year":"2013","journal-title":"Interspeech"},{"key":"ref_42","unstructured":"Ekbal, A., Haque, R., and Bandyopadhyay, S. (2008, January 7\u201312). Named entity recognition in Bengali: A conditional random field approach. Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II, Hyderabad, India."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Fu, J., Liu, X., and Huang, X. (2018, January 2\u20137). Adaptive co-attention network for named entity recognition in tweets. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11962"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Ma, X., and Hovy, E. (2016). End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv.","DOI":"10.18653\/v1\/P16-1101"},{"key":"ref_45","unstructured":"Lafferty, J.D., McCallum, A., and Pereira, F.C.N. (July, January 28). Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, USA."},{"key":"ref_46","unstructured":"Zhu, X.J. (2005). Semi-Supervised Learning Literature Survey, University of Wisconsin-Madison."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"221","DOI":"10.31449\/inf.v43i2.2217","article-title":"A new ensemble semi-supervised self-labeled algorithm","volume":"43","author":"Livieris","year":"2019","journal-title":"Informatica"},{"key":"ref_48","unstructured":"Yarowsky, D. (August, January 30). Unsupervised word sense disambiguation rivaling supervised methods. Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Didaci, L., and Roli, F. (2006, January 17\u201319). Using co-training and self-training in semi-supervised multiple classifier systems. Proceedings of the Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), Hong Kong, China.","DOI":"10.1007\/11815921_57"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rustam, F., Ashraf, I., Mehmood, A., Ullah, S., and Choi, G.S. (2019). Tweets Classification on the Base of Sentiments for US Airline Companies. Entropy, 21.","DOI":"10.3390\/e21111078"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1093\/jamia\/ocu041","article-title":"Pharmacovigilance from social media: Mining adverse drug reaction mentions using sequence labeling with word embedding cluster features","volume":"22","author":"Nikfarjam","year":"2015","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1093\/jamia\/ocx045","article-title":"Mining e-cigarette adverse events in social media using Bi-LSTM recurrent neural network with word embedding representation","volume":"25","author":"Xie","year":"2017","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Luan, Y., Wadden, D., He, L., Shah, A., Ostendorf, M., and Hajishirzi, H. (2019). A general framework for information extraction using dynamic span graphs. arXiv.","DOI":"10.18653\/v1\/N19-1308"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Salazar, A., Safont, G., and Vergara, L. (2018, January 8\u201313). Semi-supervised learning for imbalanced classification of credit card transactions. Proceedings of the 2018 IEEE International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489755"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Zhu, X., and Goldberg, A.B. (2009). Introduction to Semi-Supervised Learning: Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan & Claypool Publisher.","DOI":"10.1007\/978-3-031-01548-9"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1177\/0735633117752614","article-title":"Predicting secondary school students\u2019 performance utilizing a semi-supervised learning approach","volume":"57","author":"Livieris","year":"2019","journal-title":"J. Educ. Comput. Res."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/252\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:00:08Z","timestamp":1760173208000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/252"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,22]]},"references-count":56,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["e22020252"],"URL":"https:\/\/doi.org\/10.3390\/e22020252","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,22]]}}}