{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,3]],"date-time":"2025-07-03T15:15:00Z","timestamp":1751555700148,"version":"3.37.3"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"22","license":[{"start":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T00:00:00Z","timestamp":1694649600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T00:00:00Z","timestamp":1694649600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2023,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The effectiveness of natural language processing models relies on various factors, including the architecture, number of parameters, data used during training, and the tasks they were trained on. Recent studies indicate that models pre-trained on large corpora and fine-tuned on task-specific datasets, covering multiple tasks, can generate remarkable results across various benchmarks. We propose a new approach based on a straightforward hypothesis: improving model performance on a target task by considering other artificial tasks defined on the same training dataset. By doing so, the model can gain further insights into the training dataset and attain a greater understanding, improving efficiency on the target task. This approach differs from others that consider multiple pre-existing tasks on different datasets. We validate this hypothesis by focusing on the problem of answering yes\/no questions and introducing a multi-task model that outputs a span of the reference text, serving as evidence for answering the question. The task of span extraction is an artificial one, designed to benefit the performance of the model answering yes\/no questions. We acquire weak supervision for these spans, by using a pre-trained extractive question answering model, dispensing the need for costly human annotation. Our experiments, using modern transformer-based language models, demonstrate that this method outperforms the standard approach of training models to answer yes\/no questions. Although the primary objective was to enhance the performance of the model in answering yes\/no questions, it was discovered that span texts are a significant source of information. These spans, derived from the question reference texts, provided valuable insights for the users to better comprehend the answers to the questions. The model\u2019s improved accuracy in answering yes\/no questions, coupled with the supplementary information provided by the span texts, led to a more comprehensive and informative user experience.<\/jats:p>","DOI":"10.1007\/s10489-023-04751-w","type":"journal-article","created":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T14:08:52Z","timestamp":1694700532000},"page":"27560-27570","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Enhancing yes\/no question answering with weak supervision via extractive question answering"],"prefix":"10.1007","volume":"53","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9404-0331","authenticated-orcid":false,"given":"Dimitris","family":"Dimitriadis","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Grigorios","family":"Tsoumakas","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,9,14]]},"reference":[{"key":"4751_CR1","unstructured":"Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21:1\u201367"},{"key":"4751_CR2","doi-asserted-by":"crossref","unstructured":"Wang A, Singh A, Michael J, Hill F, Levy O (2018) Bowman SR Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv:1804.07461","DOI":"10.18653\/v1\/W18-5446"},{"key":"4751_CR3","doi-asserted-by":"crossref","unstructured":"Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. arXiv:1606.05250","DOI":"10.18653\/v1\/D16-1264"},{"key":"4751_CR4","doi-asserted-by":"crossref","unstructured":"Rajpurkar P, Jia R, Liang P (2018) Know what you don\u2019t know: Unanswerable. arXiv:1806.03822","DOI":"10.18653\/v1\/P18-2124"},{"key":"4751_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.eng.2022.04.024","author":"H Wang","year":"2022","unstructured":"Wang H, Li J, Wu H, Hovy E, Sun Y (2022) Pre-trained language models and their applications. Eng. https:\/\/doi.org\/10.1016\/j.eng.2022.04.024","journal-title":"Eng"},{"key":"4751_CR6","doi-asserted-by":"crossref","unstructured":"Strubell E, Ganesh A, McCallum A (2019) Energy and policy considerations for deep learning in nlp. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3645\u20133650","DOI":"10.18653\/v1\/P19-1355"},{"key":"4751_CR7","doi-asserted-by":"crossref","unstructured":"Strubell E, Ganesh A, McCallum A (2020) Energy and policy considerations for modern deep learning research. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp 13693\u201313696","DOI":"10.1609\/aaai.v34i09.7123"},{"key":"4751_CR8","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference 1(Mlm), 4171\u20134186. arXiv:1810.04805"},{"key":"4751_CR9","doi-asserted-by":"crossref","unstructured":"Nishida K, Nishida K, Nagata M, Otsuka A, Saito I, Asano H, Tomita J (2019) Answering while summarizing: Multi-task learning for\u00a0multihop qa with evidence extraction. arXiv:1905.08511","DOI":"10.18653\/v1\/P19-1225"},{"key":"4751_CR10","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692"},{"key":"4751_CR11","doi-asserted-by":"crossref","unstructured":"Luo M, Chen S, Baral, C (2021) A simple approach to jointly rank passages and select relevant sentences in the obqa context. arXiv:2109.10497","DOI":"10.18653\/v1\/2022.naacl-srw.23"},{"key":"4751_CR12","doi-asserted-by":"publisher","unstructured":"Clark C, Lee K, Chang MW, Kwiatkowski T, Collins M, Toutanova K (2019) BoolQ: Exploring the surprising difficulty of natural yes\/no questions. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp 2924\u20132936. Association for Computational Linguistics, Minneapolis, Minnesota. https:\/\/doi.org\/10.18653\/v1\/N19-1300. https:\/\/aclanthology.org\/N19-1300","DOI":"10.18653\/v1\/N19-1300"},{"key":"4751_CR13","unstructured":"Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z, et al. (2023) A survey of large language models. arXiv:2303.18223"},{"key":"4751_CR14","unstructured":"Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2020) Albert: A lite bert for self-supervised learning of language representations. In: International Conference on Learning Representations. https:\/\/openreview.net\/pdf?id=H1eA7AEtvS"},{"key":"4751_CR15","doi-asserted-by":"publisher","unstructured":"Paramasivam A, Nirmala SJ (2022) A survey on textual entailment based question answering. J King Saud University - Comput Inf Sci 34(10, Part B):9644\u20139653. https:\/\/doi.org\/10.1016\/j.jksuci.2021.11.017","DOI":"10.1016\/j.jksuci.2021.11.017"},{"key":"4751_CR16","unstructured":"Wang A, Pruksachatkun Y, Nangia N, Singh A, Michael J, Hill F, Levy O, Bowman S (2019) Superglue: A stickier benchmark for generalpurpose language understanding systems. Adv Neural Inf Process Syst 32"},{"key":"4751_CR17","unstructured":"Phang, J., F\u00e9vry, T., Bowman, S.R (2018) Sentence encoders on stilts: Supplementary training on intermediate labeled-data tasks. arXiv:1811.01088"},{"key":"4751_CR18","unstructured":"He P, Liu X, Gao J, Chen W (2021) Deberta: Decoding-enhanced bert with disentangled attention. In: International Conference on Learning Representations. https:\/\/openreview.net\/pdf?id=XPZIaotutsD"},{"key":"4751_CR19","doi-asserted-by":"crossref","unstructured":"Tam D, Menon RR, Bansal M, Srivastava S, Raffel C (2021) Improving and simplifying pattern exploiting training. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp 4980\u20134991","DOI":"10.18653\/v1\/2021.emnlp-main.407"},{"key":"4751_CR20","doi-asserted-by":"publisher","unstructured":"Tam D, R Menon R, Bansal M, Srivastava S, Raffel C (2021) Improving and simplifying pattern exploiting training. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 4980\u20134991. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic. https:\/\/doi.org\/10.18653\/v1\/2021.emnlp-main.407. https:\/\/aclanthology.org\/2021.emnlp-main.407","DOI":"10.18653\/v1\/2021.emnlp-main.407"},{"key":"4751_CR21","unstructured":"Wei J, Bosma M, Zhao V, Guu K, Yu AW, Lester B, Du N, Dai AM, Le QV (2022) Finetuned language models are zero-shot learners. In: International Conference on Learning Representations"},{"key":"4751_CR22","unstructured":"Wang S, Fang H, Khabsa M, Mao H, Ma H (2021) Entailment as few-shot learner. arXiv:2104.14690"},{"key":"4751_CR23","unstructured":"Wu S, Irsoy O, Lu S, Dabravolski V, Dredze M, Gehrmann S, Kambadur P, Rosenberg D, Mann G (2023) Bloomberggpt: A large language model for finance. arXiv:2303.17564"},{"key":"4751_CR24","doi-asserted-by":"crossref","unstructured":"Black S, Biderman S, Hallahan E, Anthony Q, Gao L, Golding L, He H, Leahy C, McDonell K, Phang J, et al. (2022) Gpt-neox-20b: An open-source autoregressive language model. In: Proceedings of BigScience Episode# 5\u2013Workshop on Challenges & Perspectives in Creating Large Language Models, pp 95\u2013136","DOI":"10.18653\/v1\/2022.bigscience-1.9"},{"key":"4751_CR25","unstructured":"Gao L, Biderman S, Black S, Golding L, Hoppe T, Foster C, Phang J, He H, Thite A, Nabeshima N, et al. (2020) The pile: An 800gb dataset of diverse text for language modeling. arXiv:2101.00027"},{"key":"4751_CR26","unstructured":"Poli M, Massaroli S, Nguyen E, Fu DY, Dao T, Baccus S, Bengio Y, Ermon S, R\u00e9 C (2023) Hyena hierarchy: Towards larger convolutional language models. arXiv:2302.10866"},{"key":"4751_CR27","unstructured":"Roy A, Anil R, Lai G, Lee B, Zhao J, Zhang S, Wang S, Zhang Y, Wu S, Swavely R, et al. (2022) N-grammer: Augmenting transformers with latent n-grams. arXiv:2207.06366"},{"key":"4751_CR28","unstructured":"Arora S, Narayan A, Chen MF, Orr LJ, Guha N, Bhatia K, Chami I, Sala F, R\u00e9 C (2022) Ask me anything: A simple strategy for prompting language models. arXiv:2210.02441"},{"key":"4751_CR29","unstructured":"Soltan S, Ananthakrishnan S, FitzGerald J, Gupta R, Hamza W, Khan H, Peris C, Rawls S, Rosenbaum A, Rumshisky A, et al. (2022) Alexatm 20b: Few-shot learning using a large-scale multilingual seq2seq model. arXiv:2208.01448"},{"key":"4751_CR30","unstructured":"Touvron H, Lavril T, Izacard G, Martinet X, Lachaux MA, Lacroix T, Rozi\u00e8re B, Goyal N, Hambro E, Azhar F, et al. (2023) Llama: Open and efficient foundation language models. arXiv:2302.13971"},{"key":"4751_CR31","unstructured":"Jurafsky D, Martin JH (2022) Speech and Language Processing (3rd ed.draft). unpublished"},{"key":"4751_CR32","unstructured":"Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al. (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32"},{"key":"4751_CR33","doi-asserted-by":"crossref","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, et al. (2019) Huggingface\u2019s transformers: State-of-the-art natural language processing. arXiv:1910.03771","DOI":"10.18653\/v1\/2020.emnlp-demos.6"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-04751-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-023-04751-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-023-04751-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,25]],"date-time":"2023-10-25T15:22:57Z","timestamp":1698247377000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-023-04751-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,14]]},"references-count":33,"journal-issue":{"issue":"22","published-print":{"date-parts":[[2023,11]]}},"alternative-id":["4751"],"URL":"https:\/\/doi.org\/10.1007\/s10489-023-04751-w","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"type":"print","value":"0924-669X"},{"type":"electronic","value":"1573-7497"}],"subject":[],"published":{"date-parts":[[2023,9,14]]},"assertion":[{"value":"30 May 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 September 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"No Funding and no conflicts of Interests\/Competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Funding and\/or Conflicts of Interests\/Competing interests"}}]}}