{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T01:39:14Z","timestamp":1777340354041,"version":"3.51.4"},"reference-count":94,"publisher":"Cambridge University Press (CUP)","issue":"3","license":[{"start":{"date-parts":[[2022,7,11]],"date-time":"2022-07-11T00:00:00Z","timestamp":1657497600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Natural language processing (NLP) technologies and applications in legal text processing are gaining momentum. Being one of the most prominent tasks in NLP, named-entity recognition (NER) can substantiate a great convenience for NLP in law due to the variety of named entities in the legal domain and their accentuated importance in legal documents. However, domain-specific NER models in the legal domain are not well studied. We present a NER model for Turkish legal texts with a custom-made corpus as well as several NER architectures based on conditional random fields and bidirectional long-short-term memories (BiLSTMs) to address the task. We also study several combinations of different word embeddings consisting of GloVe, Morph2Vec, and neural network-based character feature extraction techniques either with BiLSTM or convolutional neural networks. We report 92.27% F1 score with a hybrid word representation of GloVe and Morph2Vec with character-level features extracted with BiLSTM. Being an agglutinative language, the morphological structure of Turkish is also considered. To the best of our knowledge, our work is the first legal domain-specific NER study in Turkish and also the first study for an agglutinative language in the legal domain. Thus, our work can also have implications beyond the Turkish language.<\/jats:p>","DOI":"10.1017\/s1351324922000304","type":"journal-article","created":{"date-parts":[[2022,7,11]],"date-time":"2022-07-11T03:36:11Z","timestamp":1657510571000},"page":"615-642","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":26,"title":["Named-entity recognition in Turkish legal texts"],"prefix":"10.1017","volume":"29","author":[{"given":"Can","family":"\u00c7etinda\u011f","sequence":"first","affiliation":[]},{"given":"Berkay","family":"Yaz\u0131c\u0131o\u011flu","sequence":"additional","affiliation":[]},{"given":"Aykut","family":"Ko\u00e7","sequence":"additional","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2022,7,11]]},"reference":[{"key":"S1351324922000304_ref37","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"S1351324922000304_ref21","doi-asserted-by":"crossref","unstructured":"Dalk\u0131l\u0131\u00e7, F.E. , Geli\u015fli, S. and Diri, B. (2010). Named entity recognition from Turkish texts. In 2010 IEEE 18th Signal Processing and Communications Applications Conference. IEEE, pp. 918\u2013920.","DOI":"10.1109\/SIU.2010.5653553"},{"key":"S1351324922000304_ref28","doi-asserted-by":"publisher","DOI":"10.3115\/1620754.1620802"},{"key":"S1351324922000304_ref48","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2011.08.131"},{"key":"S1351324922000304_ref19","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"S1351324922000304_ref22","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2014.24"},{"key":"S1351324922000304_ref66","unstructured":"Nanda, R. , Adebayo, K.J. , Di Caro, L. , Boella, G. and Robaldo, L. (2017). Legal information retrieval using topic clustering and neural networks. In COLIEE@ ICAIL, pp. 68\u201378."},{"key":"S1351324922000304_ref77","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324922000122"},{"key":"S1351324922000304_ref82","doi-asserted-by":"crossref","unstructured":"Strakov\u00e1, J. , Straka, M. and Haji\u010d, J. (2019). Neural architectures for nested NER through linearization. arXiv preprint arXiv:1908.06926.","DOI":"10.18653\/v1\/P19-1527"},{"key":"S1351324922000304_ref92","doi-asserted-by":"crossref","unstructured":"Yamada, I. , Asai, A. , Shindo, H. , Takeda, H. and Matsumoto, Y. (2020). Luke: Deep contextualized entity representations with entity-aware self-attention. arXiv preprint arXiv:2010.01057.","DOI":"10.18653\/v1\/2020.emnlp-main.523"},{"key":"S1351324922000304_ref8","unstructured":"Borthwick, A. and Grishman, R. (1999). A Maximum Entropy Approach to Named Entity Recognition. PhD Thesis, Citeseer."},{"key":"S1351324922000304_ref55","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-33709-4_21"},{"key":"S1351324922000304_ref51","doi-asserted-by":"crossref","unstructured":"Lample, G. , Ballesteros, M. , Subramanian, S. , Kawakami, K. and Dyer, C. (2016). Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360.","DOI":"10.18653\/v1\/N16-1030"},{"key":"S1351324922000304_ref16","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3936759"},{"key":"S1351324922000304_ref53","first-page":"272","volume-title":"International Conference on Semantic Systems","author":"Leitner","year":"2019"},{"key":"S1351324922000304_ref63","unstructured":"Mikolov, T. , Chen, K. , Corrado, G. and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781."},{"key":"S1351324922000304_ref72","doi-asserted-by":"crossref","unstructured":"Peters, M.E. , Neumann, M. , Iyyer, M. , Gardner, M. , Clark, C. , Lee, K. and Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365.","DOI":"10.18653\/v1\/N18-1202"},{"key":"S1351324922000304_ref34","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324918000281"},{"key":"S1351324922000304_ref36","doi-asserted-by":"publisher","DOI":"10.3115\/1119176.1119202"},{"key":"S1351324922000304_ref67","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-018-9225-1"},{"key":"S1351324922000304_ref44","doi-asserted-by":"publisher","DOI":"10.1109\/ACDT.2018.8592948"},{"key":"S1351324922000304_ref71","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"S1351324922000304_ref18","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00104"},{"key":"S1351324922000304_ref25","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/E14-2001"},{"key":"S1351324922000304_ref83","unstructured":"\u015eulea, O. , Zampieri, M. , Malmasi, S. , Vela, M. , Dinu, L.P. and van Genabith, J. (2017a). Exploring the use of text classification in the legal domain. In Ashley K.D., Atkinson K., Branting L.K., Francesconi, E., Grabmair, M., Lauritsen M., Walker V.R. and Wyner A.Z. (eds), Proceedings of the Second Workshop on Automated Semantic Analysis of Information in Legal Texts Co-located with the 16th International Conference on Artificial Intelligence and Law (ICAIL 2017), London, UK, June 16, 2017. CEUR Workshop Proceedings, vol. 2143. CEUR-WS.org."},{"key":"S1351324922000304_ref87","doi-asserted-by":"crossref","unstructured":"\u00dcst\u00fcn, A. , Kurfal, M. and Can, B. (2018). Characters or morphemes: How to represent words? In Proceedings of The Third Workshop on Representation Learning for NLP, pp. 144\u2013153.","DOI":"10.18653\/v1\/W18-3019"},{"key":"S1351324922000304_ref88","doi-asserted-by":"crossref","unstructured":"Vardhan, H. , Surana, N. and Tripathy, B. (2020). Named-entity recognition for legal documents. In International Conference on Advanced Machine Learning Technologies and Applications. Springer, pp. 469\u2013479.","DOI":"10.1007\/978-981-15-3383-9_43"},{"key":"S1351324922000304_ref24","doi-asserted-by":"publisher","DOI":"10.1145\/3299819.3299846"},{"key":"S1351324922000304_ref73","doi-asserted-by":"publisher","DOI":"10.2307\/4099370"},{"key":"S1351324922000304_ref75","doi-asserted-by":"publisher","DOI":"10.1109\/ICoAC.2017.7951772"},{"key":"S1351324922000304_ref49","unstructured":"Kuru, O. , Can, O.A. and Yuret, D. (2016). CharNER: Character-level named entity recognition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 911\u2013921."},{"key":"S1351324922000304_ref26","doi-asserted-by":"publisher","DOI":"10.1017\/S135132490600427X"},{"key":"S1351324922000304_ref12","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-45960-7"},{"key":"S1351324922000304_ref27","doi-asserted-by":"publisher","DOI":"10.3115\/1219840.1219885"},{"key":"S1351324922000304_ref91","unstructured":"Yadav, V. and Bethard, S. (2019). A survey on recent advances in named entity recognition from deep learning models. arXiv preprint arXiv:1910.11470."},{"key":"S1351324922000304_ref2","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324919000627"},{"key":"S1351324922000304_ref6","doi-asserted-by":"publisher","DOI":"10.1145\/2425327.2425330"},{"key":"S1351324922000304_ref80","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2018.00022"},{"key":"S1351324922000304_ref65","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2021.102684"},{"key":"S1351324922000304_ref38","unstructured":"Huang, Z. , Xu, W. and Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991."},{"key":"S1351324922000304_ref15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1636"},{"key":"S1351324922000304_ref47","author":"K\u00fc\u00e7\u00fck","year":"2009"},{"key":"S1351324922000304_ref1","unstructured":"Akdemir, A. (2018). Named Entity Recognition in Turkish Using Deep Learning Methods and Joint Learning. PhD Thesis, Bogazi\u00e7i University."},{"key":"S1351324922000304_ref3","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.93"},{"key":"S1351324922000304_ref42","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0174698"},{"key":"S1351324922000304_ref45","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220316"},{"key":"S1351324922000304_ref57","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99722-3_32"},{"key":"S1351324922000304_ref10","doi-asserted-by":"publisher","DOI":"10.2307\/1227753"},{"key":"S1351324922000304_ref40","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(03)00106-1"},{"key":"S1351324922000304_ref23","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-12837-0_2"},{"key":"S1351324922000304_ref69","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-90165-7"},{"key":"S1351324922000304_ref70","unstructured":"O\u2019Sullivan, C. and Beel, J. (2019). Predicting the outcome of judicial decisions made by the European Court of Human Rights. In Proceedings of the 27th AIAI Irish Conference on Artificial Intelligence and Cognitive Science, Dublin, Ireland."},{"key":"S1351324922000304_ref94","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-90165-7_6"},{"key":"S1351324922000304_ref13","doi-asserted-by":"publisher","DOI":"10.1145\/3086512.3086515"},{"key":"S1351324922000304_ref39","doi-asserted-by":"publisher","DOI":"10.1109\/ICDS47004.2019.8942343"},{"key":"S1351324922000304_ref78","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-017-9197-6"},{"key":"S1351324922000304_ref68","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/9.2.137"},{"key":"S1351324922000304_ref74","unstructured":"Sang, E.F. and De Meulder, F. (2003). Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050."},{"key":"S1351324922000304_ref56","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32381-3_45"},{"key":"S1351324922000304_ref20","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324918000475"},{"key":"S1351324922000304_ref84","doi-asserted-by":"publisher","DOI":"10.26615\/978-954-452-049-6_092"},{"key":"S1351324922000304_ref33","doi-asserted-by":"publisher","DOI":"10.3115\/1072399.1072401"},{"key":"S1351324922000304_ref35","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020271707826"},{"key":"S1351324922000304_ref7","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-012-9131-x"},{"key":"S1351324922000304_ref46","doi-asserted-by":"crossref","unstructured":"K\u00fc\u00e7\u00fck, D. and Steinberger, R. (2014). Experiments to improve named entity recognition on Turkish tweets. arXiv preprint arXiv:1410.8668.","DOI":"10.3115\/v1\/W14-1309"},{"key":"S1351324922000304_ref11","doi-asserted-by":"publisher","DOI":"10.1145\/3086512.3086514"},{"key":"S1351324922000304_ref41","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-017-9566-2"},{"key":"S1351324922000304_ref86","doi-asserted-by":"publisher","DOI":"10.1017\/S135132490200284X"},{"key":"S1351324922000304_ref31","unstructured":"Galgani, F. , Compton, P. and Hoffmann, A. (2012). Combining different summarization techniques for legal text. In Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data, HYBRID\u201912, USA. Association for Computational Linguistics, pp. 115\u2013123."},{"key":"S1351324922000304_ref89","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2018.10348"},{"key":"S1351324922000304_ref93","unstructured":"Yeniterzi, R. (2011). Exploiting morphology in Turkish named entity recognition system. In Proceedings of the ACL 2011 Student Session, pp. 105\u2013110."},{"key":"S1351324922000304_ref4","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(03)00105-X"},{"key":"S1351324922000304_ref59","doi-asserted-by":"crossref","unstructured":"Manor, L. and Junyi, J.L. (2019). Plain English summarization of contracts. In Proceedings of the Natural Legal Language Processing Workshop 2019, pp. 1\u201311.","DOI":"10.18653\/v1\/W19-2201"},{"key":"S1351324922000304_ref54","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.2981314"},{"key":"S1351324922000304_ref17","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-018-9238-9"},{"key":"S1351324922000304_ref76","unstructured":"\u015eeker, G.A. and Eryi\u011fit, G. (2012). Initial explorations on using CRFs for Turkish named entity recognition. In Proceedings of COLING 2012, pp. 2459\u20132474."},{"key":"S1351324922000304_ref43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-50953-2_20"},{"key":"S1351324922000304_ref90","doi-asserted-by":"crossref","unstructured":"Wang, X. , Jiang, Y. , Bach, N. , Wang, T. , Huang, Z. , Huang, F. and Tu, K. (2020). Automated Concatenation of Embeddings for Structured Prediction. arXiv preprint arXiv:2010.05006.","DOI":"10.18653\/v1\/2021.acl-long.206"},{"key":"S1351324922000304_ref60","doi-asserted-by":"publisher","DOI":"10.1017\/S1537592704040502"},{"key":"S1351324922000304_ref5","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-009-9077-9"},{"key":"S1351324922000304_ref29","volume-title":"Lecture Notes in Computer Science","author":"Francesconi","year":"2010"},{"key":"S1351324922000304_ref62","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-019-09255-y"},{"key":"S1351324922000304_ref64","unstructured":"Morimoto, A. , Kubo, D. , Sato, M. , Shindo, H. and Matsumoto, Y. (2017). Legal question answering system using neural attention. In Satoh K., Kim M., Kano Y., Goebel R. and Oliveira T. (eds), 4th Competition on Legal Information Extraction and Entailment (COLIEE 2017), Held in Conjunction with the 16th International Conference on Artificial Intelligence and Law (ICAIL 2017) in King\u2019s College London, UK. EPiC Series in Computing, vol. 47. EasyChair, pp. 79\u201389."},{"key":"S1351324922000304_ref81","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-2208"},{"key":"S1351324922000304_ref85","unstructured":"Tkachenko, M. and Simanovsky, A. (2012). Named entity recognition: Exploring features. In KONVENS, pp. 118\u2013127."},{"key":"S1351324922000304_ref50","unstructured":"Lafferty, J.D. , McCallum, A. and Pereira, F.C.N. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML\u201901, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc., pp. 282\u2013289."},{"key":"S1351324922000304_ref14","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-2209"},{"key":"S1351324922000304_ref9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-00178-0_32"},{"key":"S1351324922000304_ref52","unstructured":"Leek, T.R. (1997). Information Extraction using Hidden Markov Models. Master\u2019s Thesis, Citeseer."},{"key":"S1351324922000304_ref32","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2005.1556215"},{"key":"S1351324922000304_ref58","doi-asserted-by":"crossref","unstructured":"Ma, X. and Hovy, E. (2016). End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. arXiv preprint arXiv:1603.01354.","DOI":"10.18653\/v1\/P16-1101"},{"key":"S1351324922000304_ref30","unstructured":"Freitag, D. and McCallum, A. (1999). Information extraction with HMMs and shrinkage. In Proceedings of the AAAI-99 Workshop on Machine Learning for Information Extraction. Orlando, Florida, pp. 31\u201336."},{"key":"S1351324922000304_ref61","unstructured":"McCallum, A. , Freitag, D. and Pereira, F.C.N. (2000). Maximum entropy markov models for information extraction and segmentation. In Proceedings of the 17th International Conference on Machine Learning, ICML\u201900, San Francisco, CA, USA, pp. 591\u2013598."},{"key":"S1351324922000304_ref79","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-2203"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324922000304","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,19]],"date-time":"2023-05-19T07:31:34Z","timestamp":1684481494000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324922000304\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,11]]},"references-count":94,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["S1351324922000304"],"URL":"https:\/\/doi.org\/10.1017\/s1351324922000304","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,11]]},"assertion":[{"value":"\u00a9 The Author(s), 2022. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}}]}}