{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:05:17Z","timestamp":1750309517956,"version":"3.41.0"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T00:00:00Z","timestamp":1737331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2025,1,31]]},"abstract":"<jats:p>Counterfactual statements are examples of causal reasoning as they describe events that did not happen and, optionally, those events\u2019 consequences if they happened. SemEval-2020 introduces the counterfactual detection (CFD) task and shares an English dataset. Since then, a set of datasets has been released in English, German, and Japanese as part of Amazon product reviews. This work releases the first Turkish corpus of counterfactuals (TRCD). The data collection process is driven by a clue phrase list of counterfactuals, mainly in the form of verb inflections in Turkish. We use clue phrase-based filtering to collect sentences from the Turkish National Corpus (TNC). On the other hand, half of the collection is subject to random word filtering to avoid selection bias due to clue phrases. After the human annotation process with an Inter Annotator Agreement of 0.65, we have 5000 sentences, of which 12.8% contain counterfactual statements. Furthermore, we provide a comprehensive baseline of transformer-based models by testing the effect of clue phrases, cross-lingual performance comparisons using the available CFD datasets, and zero-shot cross-lingual classification experiments using fine-tuning on the different combinations of the existing datasets. The results confirm that TRCD is compatible with the other CFD datasets. Moreover, fine-tuning a Turkish-specific model (BERTurk) performs better than the multilingual alternatives (mBERT and XLM-R). BERTurk is more robust to clue phrase masking. This result emphasizes the importance of a language-specific tokenizer for contextual understanding, especially for low-resource languages. Finally, our qualitative analysis gives insights into errors by different models.<\/jats:p>","DOI":"10.1145\/3706105","type":"journal-article","created":{"date-parts":[[2024,11,27]],"date-time":"2024-11-27T09:51:48Z","timestamp":1732701108000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Recognition of Counterfactual Statements in Turkish"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0706-0412","authenticated-orcid":false,"given":"Ali","family":"Acar","sequence":"first","affiliation":[{"name":"Izmir Institute of Technology, Urla, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0488-9682","authenticated-orcid":false,"given":"Selma","family":"Tekir","sequence":"additional","affiliation":[{"name":"Izmir Institute of Technology, Urla, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,1,20]]},"reference":[{"key":"e_1_3_4_2_2","unstructured":"Mustafa Aksan Ahmet Koltuksuz Taner Sezer \u00dcmit Mersinli Umut Ufuk Demirhan Hakan Yilmazer G\u00fcls\u00fcm Atasoy Seda \u00d6z Ipek Yildiz and \u00d6zlem Kurtoglu. 2012. Construction of the Turkish National Corpus (TNC). In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC 2012) \u0130stanbul Turkey. Retrieved from http:\/\/www.lrec-conf.org\/proceedings\/lrec2012\/pdf\/991_Paper.pdf"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.82"},{"key":"e_1_3_4_4_2","volume-title":"Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit","author":"Bird Steven","year":"2009","unstructured":"Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O\u2019Reilly Media, Inc.."},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.662"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/SIU49456.2020.9302249"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S17-2001"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.84"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","unstructured":"Davide Chicco and Giuseppe Jurman. 2020. The advantages of the matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21 6 (Jan2020). DOI:10.1186\/s12864-019-6413-7","DOI":"10.1186\/s12864-019-6413-7"},{"key":"e_1_3_4_10_2","first-page":"6174","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"\u00c7\u00f6ltekin \u00c7a\u011fr\u0131","year":"2020","unstructured":"\u00c7a\u011fr\u0131 \u00c7\u00f6ltekin. 2020. A corpus of turkish offensive language on social media. In Proceedings of the 12th Language Resources and Evaluation Conference. Nicoletta Calzolari, Fr\u00e9d\u00e9ric B\u00e9chet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, H\u00e9l\u00e8ne Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association, Marseille, France, 6174\u20136184. Retrieved from https:\/\/aclanthology.org\/2020.lrec-1.758"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","unstructured":"\u00c7a\u011fr\u0131 \u00c7\u00f6ltekin A. Seza Do\u011fru\u00f6z and \u00d6zlem \u00c7etino\u011flu. 2023. Resources for turkish natural language processing: A critical survey. Language Resources and Evaluation 57 1 (01 Mar2023) 449\u2013488. DOI:10.1007\/s10579-022-09605-4","DOI":"10.1007\/s10579-022-09605-4"},{"key":"e_1_3_4_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1269"},{"key":"e_1_3_4_14_2","unstructured":"Tom Costello and John McCarthy. 1999. Useful counterfactuals. electronic transactions on artificial intelligence. Electronic Transactions on Artificial Intelligence 3 2 (1999). Retrieved from http:\/\/jmc.stanford.edu\/articles\/counterfactuals\/counterfactuals.pdf"},{"key":"e_1_3_4_15_2","volume-title":"Counterfactuality in Turkish","author":"Denizer Faik Utkan","year":"2023","unstructured":"Faik Utkan Denizer. 2023. Counterfactuality in Turkish. PhD Dissertation. Hacettepe University, Ankara, Turkey. Available at https:\/\/www.openaccess.hacettepe.edu.tr\/xmlui\/bitstream\/handle\/11655\/29566\/10509426-Denizer.pdf."},{"key":"e_1_3_4_16_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423"},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.43"},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.53"},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","unstructured":"JL Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76 5 (November1971) 378\u2013382. DOI:10.1037\/h0031619","DOI":"10.1037\/h0031619"},{"key":"e_1_3_4_20_2","volume-title":"An Enquiry Concerning Human Understanding","author":"Hume David","year":"2007","unstructured":"David Hume and P. F. Millican. 2007. An Enquiry Concerning Human Understanding. Oxford University Press, Oxford."},{"key":"e_1_3_4_21_2","volume-title":"Counterfactuals in the Language of Social Media: A Natural Language Processing Project in Conjunction with the World Well Being Project.","author":"Janocko Anthony","year":"2016","unstructured":"Anthony Janocko, Allegra Larche, Joseph Raso, and Kevin Zembrroski. 2016. Counterfactuals in the Language of Social Media: A Natural Language Processing Project in Conjunction with the World Well Being Project.Technical report. University of Pennsylvania."},{"key":"e_1_3_4_22_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Lan Zhenzhong","year":"2020","unstructured":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=H1eA7AEtvS"},{"key":"e_1_3_4_23_2","first-page":"646","volume-title":"Proceedings of the 23rd AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008","author":"Larochelle Hugo","year":"2008","unstructured":"Hugo Larochelle, Dumitru Erhan, and Yoshua Bengio. 2008. Zero-data learning of new tasks. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008. Dieter Fox and Carla P. Gomes (Eds.). AAAI Press, 646\u2013651. Retrieved from http:\/\/www.aaai.org\/Library\/AAAI\/2008\/aaai08-103.php"},{"key":"e_1_3_4_24_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.85"},{"key":"e_1_3_4_26_2","doi-asserted-by":"crossref","unstructured":"Brian W. Matthews. 1975. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405 2 (1975) 442\u2013451.","DOI":"10.1016\/0005-2795(75)90109-9"},{"key":"e_1_3_4_27_2","doi-asserted-by":"crossref","unstructured":"Bella K. Milmed. 1957. Counterfactual statements and logical modality. Mind 66 264 (1957) 453\u2013470.","DOI":"10.1093\/mind\/LXVI.264.453"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","unstructured":"Thong Nguyen and Truc-My Nguyen. 2023. Topic-Aware Causal Intervention for Counterfactual Detection. In Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities Miami USA. 165\u2013176. DOI:10.36227\/techrxiv.24225808.v1","DOI":"10.36227\/techrxiv.24225808.v1"},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-010-0201-1_15"},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.56"},{"key":"e_1_3_4_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.568"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511803161"},{"key":"e_1_3_4_33_2","volume-title":"The Book of Why: The New Science of Cause and Effect (1st ed.)","author":"Pearl Judea","year":"2018","unstructured":"Judea Pearl and Dana Mackenzie. 2018. The Book of Why: The New Science of Cause and Effect (1st ed.). Basic Books, Inc., USA."},{"key":"e_1_3_4_34_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_4_35_2","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1 8 (2019) 9."},{"key":"e_1_3_4_36_2","unstructured":"Bilge Say Deniz Zeyrek Boz\u015fahin Kemal Oflazer and Umut \u00d6zge. 2002. Development of a corpus and a treebank for present-day written Turkish. In Proceedings of the 11th International Conference of Turkish Linguistics (ICTIL 2002) 183\u2013192. Retrieved from https:\/\/hdl.handle.net\/11511\/81526"},{"key":"e_1_3_4_37_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.3770924"},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-2103"},{"key":"e_1_3_4_39_2","unstructured":"Mukund Sundararajan Ankur Taly and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. In Proceedings of the 34th International Conference on Machine Learning (ICML 2017). 70:3319\u20133328. Retrieved from http:\/\/proceedings.mlr.press\/v70\/sundararajan17a.html"},{"key":"e_1_3_4_40_2","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/978-3-030-27355-2_8","volume-title":"Proceedings of the Big Data Innovations and Applications","author":"To\u00e7o\u011flu Mansur Alp","year":"2019","unstructured":"Mansur Alp To\u00e7o\u011flu and Aytu\u011f Onan. 2019. Satire detection in turkish news articles: A machine learning approach. In Proceedings of the Big Data Innovations and Applications. Muhammad Younas, Irfan Awan, and Salima Benbernou (Eds.). Springer International Publishing, Cham, 107\u2013117."},{"key":"e_1_3_4_41_2","doi-asserted-by":"publisher","unstructured":"Utku T\u00fcrk Furkan Atmaca \u015eaziye Bet\u00fcl \u00d6zate\u015f G\u00f6zde Berk Seyyit Talha Bedir Abdullatif K\u00f6ksal Balk\u0131z \u00d6zt\u00fcrk Ba\u015faran Tunga G\u00fcng\u00f6r and Arzucan \u00d6zg\u00fcr. 2022. Resources for turkish dependency parsing: Introducing the BOUN treebank and the BoAT annotation tool. Language Resources and Evaluation 56 1 (01 Mar2022) 259\u2013307. DOI:10.1007\/s10579-021-09558-0","DOI":"10.1007\/s10579-021-09558-0"},{"key":"e_1_3_4_42_2","doi-asserted-by":"crossref","first-page":"28","DOI":"10.18653\/v1\/2022.mrl-1.3","volume-title":"Proceedings of the 2nd Workshop on Multi-Lingual Representation Learning (MRL)","author":"Ushio Asahi","year":"2022","unstructured":"Asahi Ushio and Danushka Bollegala. 2022. Zero-shot cross-lingual counterfactual detection via automatic extraction and prediction of clue phrases. In Proceedings of the 2nd Workshop on Multi-Lingual Representation Learning (MRL). Association for Computational Linguistics, Abu Dhabi, United Arab Emirates (Hybrid), 28\u201337. Retrieved from https:\/\/aclanthology.org\/2022.mrl-1.3"},{"key":"e_1_3_4_43_2","unstructured":"Melike \u00dcz\u00fcm. 2020. T\u00fcrk\u00e7ede kar\u015f\u0131olgusall\u0131k: Korpus temelli bir \u0130nceleme. Dilbilimde G\u00fcncel Tart\u0131\u015fmalar (2020) 123\u2013130."},{"key":"e_1_3_4_44_2","doi-asserted-by":"crossref","unstructured":"S. Wachter B. Mittelstadt and C. Russell. 2018. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law and Technology 31 2 (2018) 841\u2013887.","DOI":"10.2139\/ssrn.3063289"},{"key":"e_1_3_4_45_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.83"},{"key":"e_1_3_4_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.semeval-1.40"},{"key":"e_1_3_4_47_2","unstructured":"Zhilin Yang Zihang Dai Yiming Yang Jaime Carbonell Russ R Salakhutdinov and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_3_4_48_2","doi-asserted-by":"publisher","unstructured":"Deniz Zeyrek I\u015f\u0131n Demir\u015fahin and Ay\u0131\u015f\u0131\u011f\u0131 B. Sevdik \u00c7all\u0131. 2013. Turkish discourse bank: Porting a discourse annotation style to a morphologically rich language. Dialogue & Discourse 4 2 (Aug.2013) 174\u2013184. DOI:10.5087\/dad.2013.208","DOI":"10.5087\/dad.2013.208"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3706105","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3706105","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:03Z","timestamp":1750295883000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3706105"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,20]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1,31]]}},"alternative-id":["10.1145\/3706105"],"URL":"https:\/\/doi.org\/10.1145\/3706105","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2025,1,20]]},"assertion":[{"value":"2023-08-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}