{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T15:28:26Z","timestamp":1776785306288,"version":"3.51.2"},"reference-count":32,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,8,6]],"date-time":"2021-08-06T00:00:00Z","timestamp":1628208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information &amp; communications Technology Promotion","doi-asserted-by":"publisher","award":["2020-0-00113"],"award-info":[{"award-number":["2020-0-00113"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The problem of measuring sentence similarity is an essential issue in the natural language processing area. It is necessary to measure the similarity between sentences accurately. Sentence similarity measuring is the task of finding semantic symmetry between two sentences, regardless of word order and context of the words. There are many approaches to measuring sentence similarity. Deep learning methodology shows a state-of-the-art performance in many natural language processing fields and is used a lot in sentence similarity measurement methods. However, in the natural language processing field, considering the structure of the sentence or the word structure that makes up the sentence is also important. In this study, we propose a methodology combined with both deep learning methodology and a method considering lexical relationships. Our evaluation metric is the Pearson correlation coefficient and Spearman correlation coefficient. As a result, the proposed method outperforms the current approaches on a KorSTS standard benchmark Korean dataset. Moreover, it performs a maximum of a 65% increase than only using deep learning methodology. Experiments show that our proposed method generally results in better performance than those with only a deep learning model.<\/jats:p>","DOI":"10.3390\/sym13081442","type":"journal-article","created":{"date-parts":[[2021,8,6]],"date-time":"2021-08-06T05:22:01Z","timestamp":1628227321000},"page":"1442","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["A Novel Hybrid Methodology of Measuring Sentence Similarity"],"prefix":"10.3390","volume":"13","author":[{"given":"Yongmin","family":"Yoo","sequence":"first","affiliation":[{"name":"AI R&D Team of AI R&D Group, NHN Diquest, Seoul 08390, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7022-4443","authenticated-orcid":false,"given":"Tak-Sung","family":"Heo","sequence":"additional","affiliation":[{"name":"AI R&D Team of AI R&D Group, NHN Diquest, Seoul 08390, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yeongjoon","family":"Park","sequence":"additional","affiliation":[{"name":"AI R&D Team of AI R&D Group, NHN Diquest, Seoul 08390, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyungsun","family":"Kim","sequence":"additional","affiliation":[{"name":"AI R&D Team of AI R&D Group, NHN Diquest, Seoul 08390, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,6]]},"reference":[{"key":"ref_1","first-page":"393","article-title":"Big data service architecture: A survey","volume":"21","author":"Wang","year":"2020","journal-title":"J. Internet Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"609","DOI":"10.32604\/iasc.2020.013939","article-title":"Research on the automatic extraction method of web data objects based on deep learning","volume":"26","author":"Peng","year":"2020","journal-title":"IASC"},{"key":"ref_3","first-page":"1065","article-title":"Context-aware collaborative filtering framework for rating prediction based on novel similarity estimation","volume":"63","author":"Ali","year":"2020","journal-title":"Comput. Mater. Contin."},{"key":"ref_4","first-page":"1033","article-title":"DL-HAR: Deep learning-based human activity recognition framework for edge computing","volume":"65","author":"Gumaei","year":"2020","journal-title":"Comput. Mater. Contin."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1126\/science.aaa8685","article-title":"Advances in natural language processing","volume":"349","author":"Hirschberg","year":"2015","journal-title":"Science"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.csl.2016.01.003","article-title":"Assessing sentence similarity through lexical, syntactic and semantic analysis","volume":"39","author":"Ferreira","year":"2016","journal-title":"Comput Speech Lang."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1138","DOI":"10.1109\/TKDE.2006.130","article-title":"Sentence similarity based on semantic nets and corpus statistics","volume":"18","author":"Li","year":"2006","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/219717.219748","article-title":"WordNet: A lexical database for English","volume":"38","author":"Miller","year":"1995","journal-title":"Commun. ACM"},{"key":"ref_9","unstructured":"Wang, Z., Mi, H., and Ittycheriah, A. (2016, January 11\u201316). Sentence similarity learning by lexical decomposition and composition. Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING 2016), Osaka, Japan."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Mueller, J., and Thyagarajan, A. (2016, January 12\u201317). Siamese recurrent architectures for learning sentence similarity. In Proceeding of the 30th AAAI Conference on Artificial Intelligence (AAAI-16), Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10350"},{"key":"ref_11","first-page":"601","article-title":"Deep feature fusion model for sentence semantic matching","volume":"61","author":"Zhang","year":"2019","journal-title":"Comput. Mater. Contin."},{"key":"ref_12","first-page":"943","article-title":"Sentence Similarity Measurement with Convolutional Neural Networks Using Semantic and Syntactic Features","volume":"63","author":"Liang","year":"2020","journal-title":"Comput. Mater. Contin."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Heo, T.S., Kim, J.D., Park, C.Y., and Kim, Y.S. (2021). Global and Local Information Adjustment for Semantic Similarity Evaluation. Appl. Sci., 11.","DOI":"10.3390\/app11052161"},{"key":"ref_14","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Farouk, M. (2019). Measuring sentences similarity: A survey. arXiv.","DOI":"10.17485\/ijst\/2019\/v12i25\/143977"},{"key":"ref_16","first-page":"437162","article-title":"A grammar-based semantic similarity algorithm for natural language sentences","volume":"2014","author":"Lee","year":"2014","journal-title":"Sci. World J."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lee, M.C., Zhang, J.W., Lee, W.X., and Ye, H.Y. (2009, January 25\u201327). Sentence similarity computation based on POS and semantic nets. Proceedings of the 2009 Fifth International Joint Conference on INC, IMS and IDC, Seoul, Korea.","DOI":"10.1109\/NCM.2009.379"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Abdalgader, K., and Skabar, A. (2010, January 7\u201310). Short-text similarity measurement using word sense disambiguation and synonym expansion. Proceedings of the Australasian Joint Conference on Artificial Intelligence, Adelaide, Australia.","DOI":"10.1007\/978-3-642-17432-2_44"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"32832","DOI":"10.4249\/scholarpedia.32832","article-title":"Deep learning","volume":"10","author":"Schmidhuber","year":"2015","journal-title":"Scholarpedia"},{"key":"ref_20","unstructured":"Pontes, E.L., Huet, S., Linhares, A.C., and Torres-Moreno, J.M. (2018). Predicting the semantic textual similarity with siamese CNN and LSTM. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"75437","DOI":"10.1109\/ACCESS.2020.2988918","article-title":"Combining Local and Global Features into a Siamese Network for Sentence Similarity","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access."},{"key":"ref_22","unstructured":"Medsker, L.R., and Jain, L.C. (2001). Recurrent neural networks. Design and Applications, CRC press. [2nd ed.]."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_24","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kenter, T., and De Rijke, M. (2015, January 19\u201323). Short text similarity with word embeddings. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Australia.","DOI":"10.1145\/2806416.2806475"},{"key":"ref_26","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5\u201310). Distributed representations of words and phrases and their compositionality. Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_27","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv."},{"key":"ref_28","unstructured":"(2021, June 21). Bert-Kor-Base. Available online: https:\/\/github.com\/kiyoungkim1\/LMkor."},{"key":"ref_29","unstructured":"(2021, June 21). Korean Raw Corpus. Available online: http:\/\/nlp.kookmin.ac.kr."},{"key":"ref_30","unstructured":"(2021, June 21). Kkma Morpheme Analyzer. Available online: http:\/\/kkma.snu.ac.kr\/."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"7735","DOI":"10.3233\/JIFS-189593","article-title":"Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure","volume":"40","author":"Heo","year":"2021","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ham, J., Choe, Y.J., Park, K., Choi, I., and Soh, H. (2020, January 16\u201320). KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, Virtual Conference.","DOI":"10.18653\/v1\/2020.findings-emnlp.39"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/8\/1442\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:41:31Z","timestamp":1760164891000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/8\/1442"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,6]]},"references-count":32,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["sym13081442"],"URL":"https:\/\/doi.org\/10.3390\/sym13081442","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,6]]}}}