{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T12:43:40Z","timestamp":1781009020182,"version":"3.54.1"},"reference-count":54,"publisher":"Cambridge University Press (CUP)","issue":"1","license":[{"start":{"date-parts":[[2022,6,30]],"date-time":"2022-06-30T00:00:00Z","timestamp":1656547200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Current research in yes\/no question answering (QA) focuses on transfer learning techniques and transformer-based models. Models trained on large corpora are fine-tuned on tasks similar to yes\/no QA, and then the captured knowledge is transferred for solving the yes\/no QA task. Most previous studies use existing similar tasks, such as natural language inference or extractive QA, for the fine-tuning step. This paper follows a different perspective, hypothesizing that an artificial yes\/no task can transfer useful knowledge for improving the performance of yes\/no QA. We introduce three such tasks for this purpose, by adapting three corresponding existing tasks: candidate answer validation, sentiment classification, and lexical simplification. Furthermore, we experimented with three different variations of the BERT model (BERT base, RoBERTa, and ALBERT). The results show that our hypothesis holds true for all artificial tasks, despite the small size of the corresponding datasets that are used for the fine-tuning process, the differences between these tasks, the decisions that we made to adapt the original ones, and the tasks\u2019 simplicity. This gives an alternative perspective on how to deal with the yes\/no QA problem, that is more creative, and at the same time more flexible, as it can exploit multiple other existing tasks and corresponding datasets to improve yes\/no QA models.<\/jats:p>","DOI":"10.1017\/s1351324922000286","type":"journal-article","created":{"date-parts":[[2022,6,30]],"date-time":"2022-06-30T04:37:47Z","timestamp":1656563867000},"page":"73-95","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":2,"title":["Artificial fine-tuning tasks for yes\/no question answering"],"prefix":"10.1017","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9404-0331","authenticated-orcid":false,"given":"Dimitris","family":"Dimitriadis","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7879-669X","authenticated-orcid":false,"given":"Grigorios","family":"Tsoumakas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"56","published-online":{"date-parts":[[2022,6,30]]},"reference":[{"key":"S1351324922000286_ref18","unstructured":"Kajiwara, T. and Komachi, M. (2016). Building a monolingual parallel corpus for text simplification using sentence similarity based on alignment between word embeddings. In COLING 2016 - 26th International Conference on Computational Linguistics, Proceedings of COLING 2016: Technical Papers, pp. 1147\u20131158."},{"key":"S1351324922000286_ref27","unstructured":"Lan, Z. , Chen, M. , Goodman, S. , Gimpel, K. , Sharma, P. and Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations."},{"key":"S1351324922000286_ref29","doi-asserted-by":"crossref","unstructured":"Magnini, B. , Negri, M. , Prevete, R. and Tanev, H. (2002). Is it the right answer? exploiting web redundancy for answer validation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 425\u2013432.","DOI":"10.3115\/1073083.1073154"},{"key":"S1351324922000286_ref1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.468"},{"key":"S1351324922000286_ref36","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"S1351324922000286_ref42","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1401"},{"key":"S1351324922000286_ref46","unstructured":"Vaswani, A. , Shazeer, N. , Parmar, N. , Uszkoreit, J. , Jones, L. , Gomez, A.N. , Kaiser, \u0141. and Polosukhin, I. (2017). Attention is all you need. In The 31st Conference on Neural Information Processing Systems (NIPS 2017)."},{"key":"S1351324922000286_ref35","first-page":"0975","article-title":"Question answering system, approaches and techniques: A review","volume":"141","author":"Pundge","year":"2016","journal-title":"International Journal of Computer Applications"},{"key":"S1351324922000286_ref39","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-2602"},{"key":"S1351324922000286_ref31","doi-asserted-by":"publisher","DOI":"10.1109\/AITB48515.2019.8947435"},{"key":"S1351324922000286_ref23","doi-asserted-by":"crossref","unstructured":"Kim, M.-Y. , Xu, Y. , Goebel, R. and Satoh, K. (2013). Answering yes\/no questions in legal bar exams. In JSAI International Symposium on Artificial Intelligence. Springer, pp. 199\u2013213.","DOI":"10.1007\/978-3-319-10061-6_14"},{"key":"S1351324922000286_ref12","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-3001"},{"key":"S1351324922000286_ref14","first-page":"1","article-title":"A survey of text question answering techniques","volume":"53","author":"Gupta","year":"2012","journal-title":"International Journal of Computer Applications"},{"key":"S1351324922000286_ref24","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2011.07.047"},{"key":"S1351324922000286_ref50","unstructured":"Xiong, C. , Zhong, V. and Socher, R. (2017). Dynamic coattention networks for question answering. In 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, pp. 1\u201314."},{"key":"S1351324922000286_ref28","unstructured":"Liu, Y. , Ott, M. , Goyal, N. , Du, J. , Joshi, M. , Chen, D. , Levy, O. , Lewis, M. , Zettlemoyer, L. and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692."},{"key":"S1351324922000286_ref25","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783380"},{"key":"S1351324922000286_ref20","unstructured":"Kano, Y. (2016). Answering yes-no questions by penalty scoring in history subjects of university entrance examinations. In Proceedings of the Open Knowledge Base and Question Answering Workshop (OKBQA 2016), pp. 91\u201396."},{"key":"S1351324922000286_ref52","doi-asserted-by":"publisher","DOI":"10.1109\/ICSESS49938.2020.9237654"},{"key":"S1351324922000286_ref54","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.11"},{"key":"S1351324922000286_ref45","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-015-0564-6"},{"key":"S1351324922000286_ref16","doi-asserted-by":"crossref","unstructured":"Ignatov, D.I. (2021). Danetqa: A yes\/no question answering dataset for the russian language. In Analysis of Images, Social Networks and Texts: 9th International Conference, AIST 2020, Skolkovo, Moscow, Russia, October 15\u201316, 2020, Revised Selected Papers, vol. 12602. Springer Nature, p. 57.","DOI":"10.1007\/978-3-030-72610-2_4"},{"key":"S1351324922000286_ref5","doi-asserted-by":"publisher","DOI":"10.5121\/ijaia.2013.4105"},{"key":"S1351324922000286_ref6","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.12.005"},{"key":"S1351324922000286_ref3","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2009.10.003"},{"key":"S1351324922000286_ref11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-43887-6_59"},{"key":"S1351324922000286_ref13","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2946594"},{"key":"S1351324922000286_ref21","unstructured":"Kano, Y. , Hoshino, R. and Taniguchi, R. (2017). Analyzable legal yes\/no question answering system using linguistic structures. In COLIEE@ ICAIL, pp. 57\u201367."},{"key":"S1351324922000286_ref4","first-page":"1","article-title":"A survey on machine reading comprehension systems","author":"Baradaran","year":"2020","journal-title":"Natural Language Engineering"},{"key":"S1351324922000286_ref17","volume-title":"Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition","author":"Jurafsky","year":"2000"},{"key":"S1351324922000286_ref9","unstructured":"Devlin, J. , Chang, M.W. , Lee, K. and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL HLT 2019-2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, vol. 1(Mlm), pp. 4171\u20134186."},{"key":"S1351324922000286_ref32","unstructured":"Pakray, P. , Bhaskar, P. , Banerjee, S. , Pal, B.C. , Bandyopadhyay, S. and Gelbukh, A.F. (2011). A hybrid question answering system based on information retrieval and answer validation. In CLEF (Notebook Papers\/Labs\/Workshop)."},{"key":"S1351324922000286_ref2","unstructured":"Alberti, C. , Lee, K. and Collins, M. (2019). A BERT baseline for the natural questions. arXiv."},{"key":"S1351324922000286_ref37","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-2124"},{"key":"S1351324922000286_ref40","doi-asserted-by":"publisher","DOI":"10.4018\/IJHISI.2017070104"},{"key":"S1351324922000286_ref41","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-2312"},{"key":"S1351324922000286_ref44","unstructured":"Socher, R. , Perelygin, A. , Wu, J. , Chuang, J. , Manning, C.D. , Ng, A.Y. and Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631\u20131642."},{"key":"S1351324922000286_ref7","doi-asserted-by":"crossref","unstructured":"Clark, C. , Lee, K. , Chang, M.W. , Kwiatkowski, T. , Collins, M. and Toutanova, K. (2019a). Boolq: Exploring the surprising difficulty of natural yes\/no questions. In NAACL HLT 2019-2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, vol. 1, pp. 2924\u20132936.","DOI":"10.18653\/v1\/N19-1300"},{"key":"S1351324922000286_ref10","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103118"},{"key":"S1351324922000286_ref49","doi-asserted-by":"crossref","unstructured":"Wolf, T. , Debut, L. , Sanh, V. , Chaumond, J. , Delangue, C. , Moi, A. , Cistac, P. , Rault, T. , Louf, R. , Funtowicz, M. , Davison, J. , Shleifer, S. , Platen, P. , Ma, C. , Jernite, Y. , Plu, J. , Xu, C. , Scao, T. , Gugger, S. , Drame, M. , Lhoest, Q. and Rush, A. (2019). Huggingface\u2019s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"S1351324922000286_ref15","first-page":"351","article-title":"Summarization of yes\/no questions using a feature function model","volume":"20","author":"He","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"S1351324922000286_ref19","unstructured":"Kanayama, H. , Usuke, Y. , Iyao, M. and Rager, J.P. (2012). Answering yes\/no questions via question inversion. In Proceedings of COLING 2012, pp. 1377\u20131392."},{"key":"S1351324922000286_ref43","unstructured":"Silpa, K.S. and Irshad, M. (2018). A survey of lexical simplification. In Emerging Trends in Engineering, Science and Technology for Society, Energy and Environment - Proceedings of the International Conference in Emerging Trends in Engineering, Science and Technology, ICETEST 2018, vol. 60, pp. 785\u2013791."},{"key":"S1351324922000286_ref33","doi-asserted-by":"publisher","DOI":"10.1145\/383952.384025"},{"key":"S1351324922000286_ref47","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-3007"},{"key":"S1351324922000286_ref48","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K17-1028"},{"key":"S1351324922000286_ref26","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2012.2184637"},{"key":"S1351324922000286_ref51","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-3104"},{"key":"S1351324922000286_ref53","unstructured":"Yu, L. , Hermann, K.M. , Blunsom, P. and Pulman, S. (2014). Deep learning for answer sentence selection. arXiv preprint arXiv:1412.1632."},{"key":"S1351324922000286_ref8","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-4828"},{"key":"S1351324922000286_ref30","doi-asserted-by":"publisher","DOI":"10.1016\/S0079-7421(08)60536-8"},{"key":"S1351324922000286_ref22","unstructured":"Kauchak, D. (2013). Improving text simplification language modeling using unsimplified text data. In ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, vol. 1, pp. 1537\u20131546."},{"key":"S1351324922000286_ref34","unstructured":"Peng, S. , You, R. , Xie, Z. , Wang, B. , Zhang, Y. and Zhu, S. (2015). The Fudan participation in the 2015 BioASQ Challenge: Large-scale biomedical semantic indexing and question answering. In CEUR Workshop Proceedings."},{"key":"S1351324922000286_ref38","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1264"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324922000286","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,28]],"date-time":"2024-09-28T03:43:55Z","timestamp":1727495035000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324922000286\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,30]]},"references-count":54,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["S1351324922000286"],"URL":"https:\/\/doi.org\/10.1017\/s1351324922000286","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,30]]},"assertion":[{"value":"\u00a9 The Author(s), 2022. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https:\/\/creativecommons.org\/licenses\/by\/4.0\/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.","name":"license","label":"License","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This content has been made available to all.","name":"free","label":"Free to read"}]}}