{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T20:58:29Z","timestamp":1768078709689,"version":"3.49.0"},"reference-count":98,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T00:00:00Z","timestamp":1631491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2021,11,30]]},"abstract":"<jats:p>Corpus preparation for low-resource languages and for development of human language technology to analyze or computationally process them is a laborious task, primarily due to the unavailability of expert linguists who are native speakers of these languages and also due to the time and resources required. Bhojpuri, Magahi, and Maithili, languages of the Purvanchal region of India (in the north-eastern parts), are low-resource languages belonging to the Indo-Aryan (or Indic) family. They are closely related to Hindi, which is a relatively high-resource language, which is why we compare them with Hindi. We collected corpora for these three languages from various sources and cleaned them to the extent possible, without changing the data in them. The text belongs to different domains and genres. We calculated some basic statistical measures for these corpora at character, word, syllable, and morpheme levels. These corpora were also annotated with parts-of-speech (POS) and chunk tags. The basic statistical measures were both absolute and relative and were expected to indicate linguistic properties, such as morphological, lexical, phonological, and syntactic complexities (or richness). The results were compared with a standard Hindi corpus. For most of the measures, we tried to match the corpus size across the languages to avoid the effect of corpus size, but in some cases it turned out that using the full corpus was better, even if sizes were very different. Although the results are not very clear, we tried to draw some conclusions about the languages and the corpora. For POS tagging and chunking, the BIS tagset was used to manually annotate the data. The POS-tagged data sizes are 16,067, 14,669, and 12,310 sentences, respectively, for Bhojpuri, Magahi, and Maithili. The sizes for chunking are 9,695 and 1,954 sentences for Bhojpuri and Maithili, respectively. The inter-annotator agreement for these annotations, using Cohen\u2019s Kappa, was 0.92, 0.64, and 0.74, respectively, for the three languages. These (annotated) corpora have been used for developing preliminary automated tools, which include POS tagger, Chunker, and Language Identifier. We have also developed the Bilingual dictionary (Purvanchal languages to Hindi) and a Synset (that can be integrated later in the Indo-WordNet) as additional resources. The main contribution of the work is the creation of basic resources for facilitating further language processing research for these languages, providing some quantitative measures about them and their similarities among themselves and with Hindi. For similarities, we use a somewhat novel measure of language similarity based on an n-gram-based language identification algorithm. An additional contribution is providing baselines for three basic NLP applications (POS tagging, chunking, and language identification) for these closely related languages.<\/jats:p>","DOI":"10.1145\/3458250","type":"journal-article","created":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T23:39:41Z","timestamp":1631576381000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Linguistic Resources for Bhojpuri, Magahi, and Maithili: Statistics about Them, Their Similarity Estimates, and Baselines for Three Applications"],"prefix":"10.1145","volume":"20","author":[{"given":"Rajesh Kumar","family":"Mundotiya","sequence":"first","affiliation":[{"name":"Indian Institute of Technology (BHU), U.P., India"}]},{"given":"Manish Kumar","family":"Singh","sequence":"additional","affiliation":[{"name":"Alexa AI, Amazon, Bangalore, Karnataka, India"}]},{"given":"Rahul","family":"Kapur","sequence":"additional","affiliation":[{"name":"Samsung Research Institute, Noida, U.P., India"}]},{"given":"Swasti","family":"Mishra","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology (BHU), India"}]},{"given":"Anil Kumar","family":"Singh","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology (BHU), U.P., India"}]}],"member":"320","published-online":{"date-parts":[[2021,9,13]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1301"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/1628960.1628985"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the Forum for Information Retrieval Evaluation (FIRE\u201914)","author":"Anand Supriya","year":"2014","unstructured":"Supriya Anand . 2014 . Language identification for transliterated forms of Indian language queries . In Proceedings of the Forum for Information Retrieval Evaluation (FIRE\u201914) . Supriya Anand. 2014. Language identification for transliterated forms of Indian language queries. In Proceedings of the Forum for Information Retrieval Evaluation (FIRE\u201914)."},{"key":"e_1_2_1_4_1","first-page":"181","article-title":"Morphology-based POS tagging on","volume":"11","author":"Badugu Srinivasu","year":"2014","unstructured":"Srinivasu Badugu . 2014 . Morphology-based POS tagging on Telugu. Int. J. Comput. Sci. Issues 11 , 1 (2014), 181 . Srinivasu Badugu. 2014. Morphology-based POS tagging on Telugu. Int. J. Comput. Sci. Issues 11, 1 (2014), 181.","journal-title":"Telugu. Int. J. Comput. Sci. Issues"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/1857999.1858026"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2824864.2824876"},{"key":"e_1_2_1_7_1","unstructured":"Akshar Bharati K. Prakash Rao Rajeev Sangal and S. M. Bendre. 2000. Basic statistical analysis of corpus and cross comparison among corpora. Technical Report Indian Institute of Information Technology.  Akshar Bharati K. Prakash Rao Rajeev Sangal and S. M. Bendre. 2000. Basic statistical analysis of corpus and cross comparison among corpora. Technical Report Indian Institute of Information Technology."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the Annual Language Testing Research Colloquium (LTRC\u201906)","author":"Bharati Akshar","year":"2006","unstructured":"Akshar Bharati , Rajeev Sangal , Dipti Misra Sharma , and Lakshmi Bai . 2006 . Anncorra: Annotating corpora guidelines for pos and chunk annotation for Indian languages . In Proceedings of the Annual Language Testing Research Colloquium (LTRC\u201906) . 1\u201338. Akshar Bharati, Rajeev Sangal, Dipti Misra Sharma, and Lakshmi Bai. 2006. Anncorra: Annotating corpora guidelines for pos and chunk annotation for Indian languages. In Proceedings of the Annual Language Testing Research Colloquium (LTRC\u201906). 1\u201338."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-5208"},{"key":"e_1_2_1_10_1","volume-title":"Indowordnet. In Proceedings of the Language Resources and Evaluation Conference (LREC\u201910)","author":"Bhattacharyya Pushpak","year":"2010","unstructured":"Pushpak Bhattacharyya . 2010 . Indowordnet. In Proceedings of the Language Resources and Evaluation Conference (LREC\u201910) . Pushpak Bhattacharyya. 2010. Indowordnet. In Proceedings of the Language Resources and Evaluation Conference (LREC\u201910)."},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the Language Resources and Evaluation Conference (LREC\u201914)","author":"Bojar Ondrej","year":"2014","unstructured":"Ondrej Bojar , Vojtech Diatka , Pavel Rychl\u1ef3 , Pavel Stran\u00e1k , V\u00edt Suchomel , Ales Tamchyna , and Daniel Zeman . 2014 . HindEnCorp-Hindi-English and Hindi-only corpus for machine translation . In Proceedings of the Language Resources and Evaluation Conference (LREC\u201914) . 3550\u20133555. Ondrej Bojar, Vojtech Diatka, Pavel Rychl\u1ef3, Pavel Stran\u00e1k, V\u00edt Suchomel, Ales Tamchyna, and Daniel Zeman. 2014. HindEnCorp-Hindi-English and Hindi-only corpus for machine translation. In Proceedings of the Language Resources and Evaluation Conference (LREC\u201914). 3550\u20133555."},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 3rd Annual Symposium on Document Analysis and Information Retrieval","volume":"161175","author":"Cavnar William B.","unstructured":"William B. Cavnar , John M. Trenkle et\u00a0al. 1994. N-gram-based text categorization . In Proceedings of the 3rd Annual Symposium on Document Analysis and Information Retrieval , Vol. 161175 . Citeseer. William B. Cavnar, John M. Trenkle et\u00a0al. 1994. N-gram-based text categorization. In Proceedings of the 3rd Annual Symposium on Document Analysis and Information Retrieval, Vol. 161175. Citeseer."},{"key":"e_1_2_1_13_1","unstructured":"Sunita Kumar Chatterji. 1926. The Evolution of Bengali Language. Rupa Delhi.  Sunita Kumar Chatterji. 1926. The Evolution of Bengali Language. Rupa Delhi."},{"key":"e_1_2_1_14_1","volume-title":"The Origin and Development of the Bengali Language","author":"Chatterji Suniti Kumar","unstructured":"Suniti Kumar Chatterji . 1986. The Origin and Development of the Bengali Language , vol. 1 . Rupa , Delhi . Suniti Kumar Chatterji. 1986. The Origin and Development of the Bengali Language, vol. 1. Rupa, Delhi."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the International Conference on Computational Linguistics, Speech, and Document Processing.","author":"Chaudhuri B. B.","unstructured":"B. B. Chaudhuri and S. Ghosh . 1998. A statistical study of Bangla corpus . In Proceedings of the International Conference on Computational Linguistics, Speech, and Document Processing. B. B. Chaudhuri and S. Ghosh. 1998. A statistical study of Bangla corpus. In Proceedings of the International Conference on Computational Linguistics, Speech, and Document Processing."},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the International Conference Recent Advances in Natural Language Processing (RANLP\u201913)","author":"Ciobanu Alina Maria","year":"2013","unstructured":"Alina Maria Ciobanu and Liviu Petrisor Dinu . 2013 . A dictionary-based approach for evaluating orthographic methods in cognates identification . In Proceedings of the International Conference Recent Advances in Natural Language Processing (RANLP\u201913) . 141\u2013147. Retrieved from https:\/\/www.aclweb.org\/anthology\/R13-1019. Alina Maria Ciobanu and Liviu Petrisor Dinu. 2013. A dictionary-based approach for evaluating orthographic methods in cognates identification. In Proceedings of the International Conference Recent Advances in Natural Language Processing (RANLP\u201913). 141\u2013147. Retrieved from https:\/\/www.aclweb.org\/anthology\/R13-1019."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 3rd Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial\u201916)","author":"\u00c7\u00f6ltekin \u00c7a\u011fr\u0131","year":"2016","unstructured":"\u00c7a\u011fr\u0131 \u00c7\u00f6ltekin and Taraka Rama . 2016 . Discriminating similar languages with linear SVMs and neural networks . In Proceedings of the 3rd Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial\u201916) . 15\u201324. \u00c7a\u011fr\u0131 \u00c7\u00f6ltekin and Taraka Rama. 2016. Discriminating similar languages with linear SVMs and neural networks. In Proceedings of the 3rd Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial\u201916). 15\u201324."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1080\/09296171003643098"},{"key":"e_1_2_1_20_1","volume-title":"Gauging similarity with n-grams: Language-independent categorization of text. Science 267, 5199","author":"Damashek Marc","year":"1995","unstructured":"Marc Damashek . 1995. Gauging similarity with n-grams: Language-independent categorization of text. Science 267, 5199 ( 1995 ), 843\u2013848. Marc Damashek. 1995. Gauging similarity with n-grams: Language-independent categorization of text. Science 267, 5199 (1995), 843\u2013848."},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the SCALLA Working Conference. 5\u20137.","author":"Dash Niladri Sekhar","year":"2004","unstructured":"Niladri Sekhar Dash . 2004 . Language corpora: Present Indian need . In Proceedings of the SCALLA Working Conference. 5\u20137. Niladri Sekhar Dash. 2004. Language corpora: Present Indian need. In Proceedings of the SCALLA Working Conference. 5\u20137."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1136"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-3911"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 456\u2013461","author":"Elfardy Heba","year":"2013","unstructured":"Heba Elfardy and Mona Diab . 2013 . Sentence-level dialect identification in Arabic . In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 456\u2013461 . Heba Elfardy and Mona Diab. 2013. Sentence-level dialect identification in Arabic. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 456\u2013461."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-2093"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.physa.2017.05.011"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of TIAD-2019 Shared Task - Translation Inference Across Dictionaries co-located with the 2nd Language, Data and Knowledge Conference (LDK\u201919)","volume":"2493","author":"Gracia Jorge","year":"2019","unstructured":"Jorge Gracia , Besim Kabashi , Ilan Kernerman , Marta Lanau-Coronas , and Dorielle Lonke . 2019 . Results of the translation inference across dictionaries 2019 shared task . In Proceedings of TIAD-2019 Shared Task - Translation Inference Across Dictionaries co-located with the 2nd Language, Data and Knowledge Conference (LDK\u201919) , Leipzig, Germany , May 20, 2019, Vol. 2493 . CEUR-WS.org, 1\u201312. Jorge Gracia, Besim Kabashi, Ilan Kernerman, Marta Lanau-Coronas, and Dorielle Lonke. 2019. Results of the translation inference across dictionaries 2019 shared task. In Proceedings of TIAD-2019 Shared Task - Translation Inference Across Dictionaries co-located with the 2nd Language, Data and Knowledge Conference (LDK\u201919), Leipzig, Germany, May 20, 2019, Vol. 2493. CEUR-WS.org, 1\u201312."},{"key":"e_1_2_1_28_1","volume-title":"Linguistic Survey of India","author":"Grierson George Abraham","unstructured":"George Abraham Grierson . 1967. Linguistic Survey of India , vol. III . Motilal Banarsidass . https:\/\/dsal.uchicago.edu\/books\/lsi\/. George Abraham Grierson. 1967. Linguistic Survey of India, vol. III. Motilal Banarsidass. https:\/\/dsal.uchicago.edu\/books\/lsi\/."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1075"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 690\u2013696","author":"Heafield Kenneth","year":"2013","unstructured":"Kenneth Heafield , Ivan Pouzyrevsky , Jonathan H. Clark , and Philipp Koehn . 2013 . Scalable modified Kneser-Ney language model estimation . In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 690\u2013696 . Retrieved from https:\/\/kheafield.com\/papers\/edinburgh\/estimate_paper.pdf. Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn. 2013. Scalable modified Kneser-Ney language model estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 690\u2013696. Retrieved from https:\/\/kheafield.com\/papers\/edinburgh\/estimate_paper.pdf."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324919000421"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1017\/S135132491900038X"},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the 2nd International Conference on Historical Linguistics","volume":"1","author":"Jeffers Robert J.","year":"1976","unstructured":"Robert J. Jeffers . 1976 . Syntactic change and syntactic reconstruction . In Proceedings of the 2nd International Conference on Historical Linguistics , vol. 1 . John Benjamin, 15. Robert J. Jeffers. 1976. Syntactic change and syntactic reconstruction. In Proceedings of the 2nd International Conference on Historical Linguistics, vol. 1. John Benjamin, 15."},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the Language Resources and Evaluation Conference (LREC\u201910)","author":"Jha Girish Nath","year":"2010","unstructured":"Girish Nath Jha . 2010 . The TDIL Program and the Indian language corpora intitiative (ILCI) . In Proceedings of the Language Resources and Evaluation Conference (LREC\u201910) . Girish Nath Jha. 2010. The TDIL Program and the Indian language corpora intitiative (ILCI). In Proceedings of the Language Resources and Evaluation Conference (LREC\u201910)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1080\/09296174.2014.911506"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSDA.2014.7051420"},{"key":"e_1_2_1_38_1","volume-title":"Kavi Narayana Murthy, and B. B. Chaudhuri","author":"Kumar G. Bharadwaja","year":"2007","unstructured":"G. Bharadwaja Kumar , Kavi Narayana Murthy, and B. B. Chaudhuri . 2007 . Statistical analysis of Telugu text corpora. International journal of Dravidian linguistics 36, 2 (2007), 71\u201399. G. Bharadwaja Kumar, Kavi Narayana Murthy, and B. B. Chaudhuri. 2007. Statistical analysis of Telugu text corpora. International journal of Dravidian linguistics 36, 2 (2007), 71\u201399."},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the International Conference on Speech and Computer (SPECOM\u201905)","author":"Kumar Rohit","unstructured":"Rohit Kumar , S. Kishore , Anumanchipalli Gopalakrishna , Rahul Chitturi , Sachin Joshi , Satinder Singh , and R. Sitaram . 2005. Development of Indian language speech databases for large vocabulary speech recognition systems . In Proceedings of the International Conference on Speech and Computer (SPECOM\u201905) . Rohit Kumar, S. Kishore, Anumanchipalli Gopalakrishna, Rahul Chitturi, Sachin Joshi, Satinder Singh, and R. Sitaram. 2005. Development of Indian language speech databases for large vocabulary speech recognition systems. In Proceedings of the International Conference on Speech and Computer (SPECOM\u201905)."},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the 5th Language and Technology Conference Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC\u201911)","author":"Kumar Ritesh","year":"2011","unstructured":"Ritesh Kumar , Bornini Lahiri , and Deepak Alok . 2011 . Challenges in developing lrs for non-scheduled languages: A case of Magahi . In Proceedings of the 5th Language and Technology Conference Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC\u201911) . 60\u201364. Ritesh Kumar, Bornini Lahiri, and Deepak Alok. 2011. Challenges in developing lrs for non-scheduled languages: A case of Magahi. In Proceedings of the 5th Language and Technology Conference Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC\u201911). 60\u201364."},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the 10th Workshop on Asian Language Resources. 105\u2013114","author":"Kumar Ritesh","year":"2012","unstructured":"Ritesh Kumar , Bornini Lahiri , and Deepak Alok . 2012 . Developing a POS tagger for Magahi: A comparative study . In Proceedings of the 10th Workshop on Asian Language Resources. 105\u2013114 . Ritesh Kumar, Bornini Lahiri, and Deepak Alok. 2012. Developing a POS tagger for Magahi: A comparative study. In Proceedings of the 10th Workshop on Asian Language Resources. 105\u2013114."},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the EUROLAN Summer School","author":"Singh Anil Kumar","year":"2007","unstructured":"Anil Kumar Singh . 2007 . Using a single framework for computational modeling of linguistic similarity for solving many NLP problems . In Proceedings of the EUROLAN Summer School . Alexandru Ioan Cuza University of Ias\u0328i. Anil Kumar Singh. 2007. Using a single framework for computational modeling of linguistic similarity for solving many NLP problems. In Proceedings of the EUROLAN Summer School. Alexandru Ioan Cuza University of Ias\u0328i."},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Lample Guillaume","year":"2018","unstructured":"Guillaume Lample , Alexis Conneau , Marc\u2019Aurelio Ranzato , Ludovic Denoyer , and Herve Jegou . 2018 . Word translation without parallel data . In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=H196sainb. Guillaume Lample, Alexis Conneau, Marc\u2019Aurelio Ranzato, Ludovic Denoyer, and Herve Jegou. 2018. Word translation without parallel data. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=H196sainb."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/E17-2020"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.02.071"},{"key":"e_1_2_1_47_1","first-page":"109","article-title":"Calculating phonological complexity","volume":"85","author":"Maddieson Ian","year":"2009","unstructured":"Ian Maddieson . 2009 . Calculating phonological complexity . Approach. Phonol. Complex. 85 (2009), 109 . Ian Maddieson. 2009. Calculating phonological complexity. Approach. Phonol. Complex. 85 (2009), 109.","journal-title":"Approach. Phonol. Complex."},{"key":"e_1_2_1_48_1","unstructured":"Khair Md Majumder and Yasir Arafat. 2006. Analysis of and observations from a Bangla News Corpus. 13\u201319. http:\/\/dspace.bracu.ac.bd\/xmlui\/handle\/10361\/616.  Khair Md Majumder and Yasir Arafat. 2006. Analysis of and observations from a Bangla News Corpus. 13\u201319. http:\/\/dspace.bracu.ac.bd\/xmlui\/handle\/10361\/616."},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the 9th European Conference on Speech Communication and Technology.","author":"Marcadet Jean-Christophe","year":"2005","unstructured":"Jean-Christophe Marcadet , Volker Fischer , and Claire Waast-Richard . 2005 . A transformation-based learning approach to language identification for mixed-lingual text-to-speech synthesis . In Proceedings of the 9th European Conference on Speech Communication and Technology. Jean-Christophe Marcadet, Volker Fischer, and Claire Waast-Richard. 2005. A transformation-based learning approach to language identification for mixed-lingual text-to-speech synthesis. In Proceedings of the 9th European Conference on Speech Communication and Technology."},{"key":"e_1_2_1_50_1","volume-title":"PAN 2017: Author profiling-gender and language variety prediction. In Proceedings of the International Conference of the Cross-Language Evaluation Forum for European Languages (CLEF\u201917)","author":"Martinc Matej","year":"2017","unstructured":"Matej Martinc , Iza Skrjanec , Katja Zupan , and Senja Pollak . 2017 . PAN 2017: Author profiling-gender and language variety prediction. In Proceedings of the International Conference of the Cross-Language Evaluation Forum for European Languages (CLEF\u201917) . Matej Martinc, Iza Skrjanec, Katja Zupan, and Senja Pollak. 2017. PAN 2017: Author profiling-gender and language variety prediction. In Proceedings of the International Conference of the Cross-Language Evaluation Forum for European Languages (CLEF\u201917)."},{"key":"e_1_2_1_51_1","volume-title":"The Indo-Aryan Languages","author":"Masica Colin P.","unstructured":"Colin P. Masica . 1993. The Indo-Aryan Languages . Cambridge University Press . Colin P. Masica. 1993. The Indo-Aryan Languages. Cambridge University Press."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.5555\/1040196.1040208"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1093\/ijl\/3.4.235"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2013.07.005"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1080\/09296170500500694"},{"key":"e_1_2_1_56_1","volume-title":"Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP\u201909)","author":"Nakov Svetlin","year":"2009","unstructured":"Svetlin Nakov , Preslav Nakov , and Elena Paskaleva . 2009 . Unsupervised extraction of false friends from parallel bi-texts using the web as a corpus . In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP\u201909) . 292\u2013298. Svetlin Nakov, Preslav Nakov, and Elena Paskaleva. 2009. Unsupervised extraction of false friends from parallel bi-texts using the web as a corpus. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP\u201909). 292\u2013298."},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC\u201916)","author":"Nasution Arbi Haza","year":"2016","unstructured":"Arbi Haza Nasution , Yohei Murakami , and Toru Ishida . 2016 . Constraint-based bilingual lexicon induction for closely related languages . In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC\u201916) . 3291\u20133298. Arbi Haza Nasution, Yohei Murakami, and Toru Ishida. 2016. Constraint-based bilingual lexicon induction for closely related languages. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC\u201916). 3291\u20133298."},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the 7th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics. 524\u2013529","author":"Ojha Atul Ku","unstructured":"Atul Ku Ojha , Pitambar Behera , Srishti Singh , and Girish N. Jha . 2015. Training & evaluation of POS taggers in Indo-Aryan languages: A case of Hindi, Odia and Bhojpuri . In Proceedings of the 7th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics. 524\u2013529 . Atul Ku Ojha, Pitambar Behera, Srishti Singh, and Girish N. Jha. 2015. Training & evaluation of POS taggers in Indo-Aryan languages: A case of Hindi, Odia and Bhojpuri. In Proceedings of the 7th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics. 524\u2013529."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-014-0585-6"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-5314"},{"key":"e_1_2_1_61_1","doi-asserted-by":"crossref","unstructured":"Nikhil Prabhu and S. Natarajan. 2019. Extraction of character personas from novels using dependency trees and POS tags. In Emerging Research in Computing Information Communication and Applications. Springer 65\u201374.  Nikhil Prabhu and S. Natarajan. 2019. Extraction of character personas from novels using dependency trees and POS tags. In Emerging Research in Computing Information Communication and Applications. Springer 65\u201374.","DOI":"10.1007\/978-981-13-5953-8_6"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csl.2019.101054"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073151"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-00382-0_29"},{"key":"e_1_2_1_65_1","first-page":"206","article-title":"Parts of speech tagger for Maithili language using HMM","volume":"7","year":"2018","unstructured":"Rishikesh. 2018 . Parts of speech tagger for Maithili language using HMM . Int. J. Innovat. Adv. Comput. Sci. 7 (2018), 206 . Rishikesh. 2018. Parts of speech tagger for Maithili language using HMM. Int. J. Innovat. Adv. Comput. Sci. 7 (2018), 206.","journal-title":"Int. J. Innovat. Adv. Comput. Sci."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2007.04.006"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11640"},{"key":"e_1_2_1_68_1","series-title":"Lecture Notes in Computer Science","volume-title":"Proceedings of the 7th International Conference on Mining Intelligence and Knowledge Exploration (MIKE\u201919)","author":"Saha Sujan Kumar","unstructured":"Sujan Kumar Saha and Ankur Priyadarshi . [n.d.]. A study on the importance of linguistic suffixes in Maithili POS tagger development . In Proceedings of the 7th International Conference on Mining Intelligence and Knowledge Exploration (MIKE\u201919) . Lecture Notes in Computer Science , vol. 11987 . Springer , 11\u201320. DOI:10.1007\/978-3-030-66187-8_2 10.1007\/978-3-030-66187-8_2 Sujan Kumar Saha and Ankur Priyadarshi. [n.d.]. A study on the importance of linguistic suffixes in Maithili POS tagger development. In Proceedings of the 7th International Conference on Mining Intelligence and Knowledge Exploration (MIKE\u201919). Lecture Notes in Computer Science, vol. 11987. Springer, 11\u201320. DOI:10.1007\/978-3-030-66187-8_2"},{"key":"e_1_2_1_69_1","volume-title":"Proceedings of the IJCAI Workshop On Shallow Parsing for South Asian Languages (SPSAL\u201907)","author":"Sangal Rajeev","year":"2007","unstructured":"Rajeev Sangal , Sushma Bendre , Dipti Sharma , and Prashanth Mannem . 2007 . Introduction to shallow parsing contest on south asian languages . In Proceedings of the IJCAI Workshop On Shallow Parsing for South Asian Languages (SPSAL\u201907) . 1\u20138. Rajeev Sangal, Sushma Bendre, Dipti Sharma, and Prashanth Mannem. 2007. Introduction to shallow parsing contest on south asian languages. In Proceedings of the IJCAI Workshop On Shallow Parsing for South Asian Languages (SPSAL\u201907). 1\u20138."},{"key":"e_1_2_1_70_1","unstructured":"A. Sarkar A. De Roeck and P. Garthwaite. 2004. Easy measures for evaluating non-English corpora for language engineering: Some lessons from Arabic and Bengali. Technical report Dept. of Comp. Faculty of Math. and Comp. Open University Walton Hall UK.  A. Sarkar A. De Roeck and P. Garthwaite. 2004. Easy measures for evaluating non-English corpora for language engineering: Some lessons from Arabic and Bengali. Technical report Dept. of Comp. Faculty of Math. and Comp. Open University Walton Hall UK."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.3115\/1073445.1073473"},{"key":"e_1_2_1_72_1","volume-title":"Advances in Computer and Computational Sciences","author":"Sharma Vijay Kumar","unstructured":"Vijay Kumar Sharma and Namita Mittal . 2018. Cross-lingual information retrieval: A dictionary-based query translation approach . In Advances in Computer and Computational Sciences . Springer , 611\u2013618. Vijay Kumar Sharma and Namita Mittal. 2018. Cross-lingual information retrieval: A dictionary-based query translation approach. In Advances in Computer and Computational Sciences. Springer, 611\u2013618."},{"key":"e_1_2_1_73_1","volume-title":"Fennig","author":"Simons Gary F.","year":"2017","unstructured":"Gary F. Simons and Charles D . Fennig . 2017 . Ethnologue : Languages of Asia. SIL International . Gary F. Simons and Charles D. Fennig. 2017. Ethnologue: Languages of Asia. SIL International."},{"key":"e_1_2_1_74_1","volume-title":"Proceedings of the 5th International Workshop on Writing Systems: Constraints on Spelling Changes. 1\u201319","author":"Singh Anil Kumar","year":"2006","unstructured":"Anil Kumar Singh . 2006 . A computational phonetic model for Indian language scripts . In Proceedings of the 5th International Workshop on Writing Systems: Constraints on Spelling Changes. 1\u201319 . Anil Kumar Singh. 2006. A computational phonetic model for Indian language scripts. In Proceedings of the 5th International Workshop on Writing Systems: Constraints on Spelling Changes. 1\u201319."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.5555\/1641976.1641985"},{"key":"e_1_2_1_76_1","volume-title":"Proceedings of the 3rd International Joint Conference on Natural Language Processing.","author":"Singh Anil Kumar","year":"2008","unstructured":"Anil Kumar Singh . 2008 . A mechanism to provide language-encoding support and an nlp friendly editor . In Proceedings of the 3rd International Joint Conference on Natural Language Processing. Anil Kumar Singh. 2008. A mechanism to provide language-encoding support and an nlp friendly editor. In Proceedings of the 3rd International Joint Conference on Natural Language Processing."},{"key":"e_1_2_1_77_1","volume-title":"Proceedings of the 3rd Web as Corpus Workshop","volume":"4","author":"Singh Anil Kumar","year":"2007","unstructured":"Anil Kumar Singh and Jagadeesh Gorla . 2007 . Identification of languages and encodings in a multilingual document . In Proceedings of the 3rd Web as Corpus Workshop , Incorporating Cleaneval: Building and Exploring Web Corpora (WAC3\u201907) , Vol. 4 . Presses Univ. de Louvain, 95. Anil Kumar Singh and Jagadeesh Gorla. 2007. Identification of languages and encodings in a multilingual document. In Proceedings of the 3rd Web as Corpus Workshop, Incorporating Cleaneval: Building and Exploring Web Corpora (WAC3\u201907), Vol. 4. Presses Univ. de Louvain, 95."},{"key":"e_1_2_1_78_1","volume-title":"Proceedings of the Language Resources and Evaluation Conference (LREC\u201908)","author":"Singh Anil Kumar","year":"2008","unstructured":"Anil Kumar Singh , Kiran Pala , and Harshit Surana . 2008 . Estimating the resource adaption cost from a resource rich language to a similar resource poor language . In Proceedings of the Language Resources and Evaluation Conference (LREC\u201908) . Anil Kumar Singh, Kiran Pala, and Harshit Surana. 2008. Estimating the resource adaption cost from a resource rich language to a similar resource poor language. In Proceedings of the Language Resources and Evaluation Conference (LREC\u201908)."},{"key":"e_1_2_1_79_1","volume-title":"Proceedings of the 26th International Conference on Computational Linguistics (COLING\u201916)","author":"Singh Loitongbam Gyanendro","year":"2016","unstructured":"Loitongbam Gyanendro Singh , Lenin Laitonjam , and Sanasam Ranbir Singh . 2016 . Automatic syllabification for manipuri language . In Proceedings of the 26th International Conference on Computational Linguistics (COLING\u201916) . 349\u2013357. Loitongbam Gyanendro Singh, Lenin Laitonjam, and Sanasam Ranbir Singh. 2016. Automatic syllabification for manipuri language. In Proceedings of the 26th International Conference on Computational Linguistics (COLING\u201916). 349\u2013357."},{"key":"e_1_2_1_80_1","volume-title":"Proceedings of the Conference on NLP, MGAHV, Wardha.","author":"Singh Srishti","unstructured":"Srishti Singh . [n.d.]. Web drawn corpus for Bhojpuri . In Proceedings of the Conference on NLP, MGAHV, Wardha. Srishti Singh. [n.d.]. Web drawn corpus for Bhojpuri. In Proceedings of the Conference on NLP, MGAHV, Wardha."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICACCI.2015.7275829"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/E14-2006"},{"key":"e_1_2_1_83_1","volume-title":"Proceedings of the 6th International Workshop on Spoken Language Technologies for Under-resourced Languages. 11\u201314","author":"Lal Brij Mohan","year":"2018","unstructured":"Brij Mohan Lal Sr ivastava, Sunayana Sitaram , Rupesh Kumar Mehta , Krishna Doss Mohan , Pallavi Matani , Sandeepkumar Satpal , Kalika Bali , Radhakrishnan Srikanth , and Niranjan Nayak . 2018 . Interspeech 2018: Low-resource automatic speech recognition challenge for Indian languages . In Proceedings of the 6th International Workshop on Spoken Language Technologies for Under-resourced Languages. 11\u201314 . Brij Mohan Lal Srivastava, Sunayana Sitaram, Rupesh Kumar Mehta, Krishna Doss Mohan, Pallavi Matani, Sandeepkumar Satpal, Kalika Bali, Radhakrishnan Srikanth, and Niranjan Nayak. 2018. Interspeech 2018: Low-resource automatic speech recognition challenge for Indian languages. In Proceedings of the 6th International Workshop on Spoken Language Technologies for Under-resourced Languages. 11\u201314."},{"key":"e_1_2_1_84_1","volume-title":"Language identification of person names using cascaded SVMs. Bachelor\u2019s Thesis","author":"Sterneberg Erik","unstructured":"Erik Sterneberg . 2012. Language identification of person names using cascaded SVMs. Bachelor\u2019s Thesis , Uppsala University , Uppsala . Erik Sterneberg. 2012. Language identification of person names using cascaded SVMs. Bachelor\u2019s Thesis, Uppsala University, Uppsala."},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1216"},{"key":"e_1_2_1_86_1","volume-title":"Proceedings of the Conference on Information and Communication Technology for Intelligent Systems (ICTIS\u201919)","author":"Zankhana","unstructured":"Zankhana B. Vaishnav and Priti S. Sajja. 2019. Knowledge-based approach for word sense disambiguation using genetic algorithm for gujarati . In Proceedings of the Conference on Information and Communication Technology for Intelligent Systems (ICTIS\u201919) . Springer, 485\u2013494. Zankhana B. Vaishnav and Priti S. Sajja. 2019. Knowledge-based approach for word sense disambiguation using genetic algorithm for gujarati. In Proceedings of the Conference on Information and Communication Technology for Intelligent Systems (ICTIS\u201919). Springer, 485\u2013494."},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1016\/0388-0001(91)90010-X"},{"key":"e_1_2_1_88_1","volume-title":"Proceedings of the Australasian Language Technology Association Workshop","volume":"12","author":"Wang Haoxing","year":"2014","unstructured":"Haoxing Wang and Laurianne Sitbon . 2014 . Multilingual lexical resources to detect cognates in non-aligned texts . In Proceedings of the Australasian Language Technology Association Workshop , Vol. 12 . 14\u201322. Haoxing Wang and Laurianne Sitbon. 2014. Multilingual lexical resources to detect cognates in non-aligned texts. In Proceedings of the Australasian Language Technology Association Workshop, Vol. 12. 14\u201322."},{"key":"e_1_2_1_89_1","volume-title":"Proceedings of the 2nd Romanian-Hungarian Joint Symposium on Applied Computational Intelligence (SACI\u201905)","author":"Windisch Gergely","year":"2005","unstructured":"Gergely Windisch and L\u00e1szl\u00f3 Csink . 2005 . Language identification using global statistics of natural languages . In Proceedings of the 2nd Romanian-Hungarian Joint Symposium on Applied Computational Intelligence (SACI\u201905) . 243\u2013255. Gergely Windisch and L\u00e1szl\u00f3 Csink. 2005. Language identification using global statistics of natural languages. In Proceedings of the 2nd Romanian-Hungarian Joint Symposium on Applied Computational Intelligence (SACI\u201905). 243\u2013255."},{"key":"e_1_2_1_90_1","volume-title":"Proceedings of the 6th Workshop on NLP for Similar Languages, Varieties and Dialects. Association for Computational Linguistics, 54\u201363","author":"Wu Nianheng","unstructured":"Nianheng Wu , Eric DeMattos , Kwok Him So , Pin-zhen Chen , and \u00c7a\u011fr\u0131 \u00c7\u00f6ltekin. 2019. Language discrimination and transfer learning for similar languages: Experiments with feature combinations and adaptation . In Proceedings of the 6th Workshop on NLP for Similar Languages, Varieties and Dialects. Association for Computational Linguistics, 54\u201363 . Retrieved from https:\/\/www.aclweb.org\/anthology\/W19-1406. Nianheng Wu, Eric DeMattos, Kwok Him So, Pin-zhen Chen, and \u00c7a\u011fr\u0131 \u00c7\u00f6ltekin. 2019. Language discrimination and transfer learning for similar languages: Experiments with feature combinations and adaptation. In Proceedings of the 6th Workshop on NLP for Similar Languages, Varieties and Dialects. Association for Computational Linguistics, 54\u201363. Retrieved from https:\/\/www.aclweb.org\/anthology\/W19-1406."},{"key":"e_1_2_1_91_1","volume-title":"Developing Linguistic Corpora: A Guide to Good Practice","author":"Wynne Martin","unstructured":"Martin Wynne . 2005. Developing Linguistic Corpora: A Guide to Good Practice . Vol. 92 . Oxbow Books Oxford . Martin Wynne. 2005. Developing Linguistic Corpora: A Guide to Good Practice. Vol. 92. Oxbow Books Oxford."},{"key":"#cr-split#-e_1_2_1_92_1.1","doi-asserted-by":"crossref","unstructured":"Yogendra P. Yadava Oliver Bond Irina Nikolaeva and Sandy Ritchie. 2019. The syntax of possessor prominence in Maithili. Prom. Intern. Possess. (2019) 39-79. DOI:10.1093\/oso\/9780198812142.003.0002 10.1093\/oso","DOI":"10.1093\/oso\/9780198812142.003.0002"},{"key":"#cr-split#-e_1_2_1_92_1.2","doi-asserted-by":"crossref","unstructured":"Yogendra P. Yadava Oliver Bond Irina Nikolaeva and Sandy Ritchie. 2019. The syntax of possessor prominence in Maithili. Prom. Intern. Possess. (2019) 39-79. DOI:10.1093\/oso\/9780198812142.003.0002","DOI":"10.1093\/oso\/9780198812142.003.0002"},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.1109\/IALP.2011.34"},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2008.2001110"},{"key":"e_1_2_1_95_1","volume-title":"Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial@COLING\u201918)","author":"Zampieri Marcos","year":"2018","unstructured":"Marcos Zampieri , Shervin Malmasi , Preslav Nakov , Ahmed Ali , Suwon Shon , James Glass , Yves Scherrer , Tanja Samard\u017ei\u0107 , Nikola Ljube\u0161i\u0107 , J\u00f6rg Tiedemann , et\u00a0al. 2018 . Language identification and morphosyntactic tagging. The second VarDial evaluation campaign . In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial@COLING\u201918) , Santa Fe, New Mexico. Association for Computational Linguistics, 1\u201317. https:\/\/aclanthology.org\/W18-3901\/. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, Suwon Shon, James Glass, Yves Scherrer, Tanja Samard\u017ei\u0107, Nikola Ljube\u0161i\u0107, J\u00f6rg Tiedemann, et\u00a0al. 2018. Language identification and morphosyntactic tagging. The second VarDial evaluation campaign. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial@COLING\u201918), Santa Fe, New Mexico. Association for Computational Linguistics, 1\u201317. https:\/\/aclanthology.org\/W18-3901\/."},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.5555\/3298023.3298059"},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.5555\/2969239.2969312"},{"key":"e_1_2_1_98_1","volume-title":"Proceedings of the 14th China Workshop on Machine Translation (CWMT\u201918)","volume":"954","author":"Zhang Yujie","year":"2019","unstructured":"Yujie Zhang . 2019 . Improving performance of NMT using semantic concept of wordnet synset . In Proceedings of the 14th China Workshop on Machine Translation (CWMT\u201918) , Vol. 954 . Springer, 39. Yujie Zhang. 2019. Improving performance of NMT using semantic concept of wordnet synset. In Proceedings of the 14th China Workshop on Machine Translation (CWMT\u201918), Vol. 954. Springer, 39."},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1163"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3458250","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3458250","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:24:39Z","timestamp":1750195479000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3458250"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,13]]},"references-count":98,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,11,30]]}},"alternative-id":["10.1145\/3458250"],"URL":"https:\/\/doi.org\/10.1145\/3458250","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,13]]},"assertion":[{"value":"2020-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}