{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T06:53:09Z","timestamp":1777704789505,"version":"3.51.4"},"reference-count":35,"publisher":"SAGE Publications","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IFS"],"published-print":{"date-parts":[[2023,11,4]]},"abstract":"<jats:p>Reading Comprehension models have achieved superhuman performance on mainstream public datasets. However, many studies have shown that the models are likely to take advantage of biases in the datasets, which makes it difficult to efficiently reasoning when generalizing to out-of-distribution datasets with non-directional bias, resulting in serious accuracy loss. Therefore, this paper proposes a pre-trained language model based de-biasing framework with positional generalization and hierarchical combination. In this work, generalized positional embedding is proposed to replace the original word embedding to initially weaken the over-dependence of the model on answer distribution information. Secondly, in order to make up for the influence of regularization randomness on training stability, KL divergence term is introduced into the loss function to constrain the distribution difference between the two sub models. Finally, a hierarchical combination method is used to obtain classification outputs that fuse text features from different encoding layers, so as to comprehensively consider the semantic features at the multidimensional level. Experimental results show that PLM-PGHC helps learn a more robust QA model and effectively restores the F1 value on the biased distribution from 37.51% to 81.78%.<\/jats:p>","DOI":"10.3233\/jifs-233029","type":"journal-article","created":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T11:23:11Z","timestamp":1693567391000},"page":"8371-8382","source":"Crossref","is-referenced-by-count":2,"title":["PLM-PGHC: A novel de-biasing framework for robust question answering"],"prefix":"10.1177","volume":"45","author":[{"given":"Shujuan","family":"Yu","sequence":"first","affiliation":[{"name":"College of Electronic and Optical Engineering & College of Flexible Electronics, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mengjie","family":"Wu","sequence":"additional","affiliation":[{"name":"College of Electronic and Optical Engineering & College of Flexible Electronics, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yun","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Electronic and Optical Engineering & College of Flexible Electronics, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Na","family":"Xie","sequence":"additional","affiliation":[{"name":"College of Electronic and Optical Engineering & College of Flexible Electronics, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liya","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Electronic and Optical Engineering & College of Flexible Electronics, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/JIFS-233029_ref1","doi-asserted-by":"crossref","first-page":"114570","DOI":"10.1016\/j.cma.2022.114570","article-title":"Dwarf mongoose optimization algorithm[J]","volume":"391","author":"Agushaka","year":"2022","journal-title":"Computer Methods in Applied Mechanics and Engineering"},{"key":"10.3233\/JIFS-233029_ref2","unstructured":"Bahdanau D. , Cho K. and Bengio Y. , Neural machine translation by jointly learning to align and translate[J], arXiv preprint arXiv:1409.0473, 2014."},{"issue":"6","key":"10.3233\/JIFS-233029_ref3","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1017\/S1351324921000395","article-title":"A survey on machine reading comprehension systems[J]","volume":"28","author":"Baradaran","year":"2022","journal-title":"Natural Language Engineering"},{"issue":"16","key":"10.3233\/JIFS-233029_ref4","doi-asserted-by":"crossref","first-page":"1929","DOI":"10.3390\/math9161929","article-title":"Hybrid fruit-fly optimization algorithm with k-means for text document clustering[J]","volume":"9","author":"Bezdan","year":"2021","journal-title":"Mathematics"},{"issue":"22","key":"10.3233\/JIFS-233029_ref5","doi-asserted-by":"crossref","first-page":"20017","DOI":"10.1007\/s00521-022-07530-9","article-title":"Prairie dog optimization algorithm[J]","volume":"34","author":"Ezugwu","year":"2022","journal-title":"Neural Computing and Applications"},{"key":"10.3233\/JIFS-233029_ref6","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1016\/j.neucom.2022.04.020","article-title":"Adversarial attack and defense technologies in natural language processing: A survey[J]","volume":"492","author":"Qiu","year":"2022","journal-title":"Neurocomputing"},{"key":"10.3233\/JIFS-233029_ref7","doi-asserted-by":"crossref","unstructured":"Clark C. , Yatskar M. and Zettlemoyer L. , Don\u2019t take the easy way out: Ensemble based methods for avoiding known dataset biases[C], Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019:4069\u20134082.","DOI":"10.18653\/v1\/D19-1418"},{"key":"10.3233\/JIFS-233029_ref8","unstructured":"Devlin J. , Chang M.W. , Lee K. et al., Bert: Pre-training of deep bidirectional transformers for language understanding[J], arXiv preprint arXiv:1810.04805, 2018."},{"key":"10.3233\/JIFS-233029_ref9","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1162\/tacl_a_00413","article-title":"Compressing large-scale transformer-based models: A case study on bert[J]","volume":"9","author":"Ganesh","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"10.3233\/JIFS-233029_ref10","doi-asserted-by":"crossref","unstructured":"Han R. , Hsu I.H. , Sun J. et al., ESTER: A machine reading comprehension dataset for reasoning about event semantic relations[C], Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021:7543\u20137559.","DOI":"10.18653\/v1\/2021.emnlp-main.597"},{"key":"10.3233\/JIFS-233029_ref11","first-page":"1693","article-title":"Teaching machines to read and comprehend[J]","volume":"28","author":"Hermann","year":"2015","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"8","key":"10.3233\/JIFS-233029_ref12","doi-asserted-by":"crossref","first-page":"1771","DOI":"10.1162\/089976602760128018","article-title":"Training products of experts by minimizing contrastive divergence[J]","volume":"14","author":"Hinton","year":"2002","journal-title":"Neural Computation"},{"issue":"3","key":"10.3233\/JIFS-233029_ref13","doi-asserted-by":"crossref","first-page":"889","DOI":"10.1007\/s13042-022-01670-z","article-title":"A novel metaheuristic optimisation approach for text sentiment analysis[J]","volume":"14","author":"Hosseinalipour","year":"2023","journal-title":"International Journal of Machine Learning and Cybernetics"},{"key":"10.3233\/JIFS-233029_ref14","first-page":"1","article-title":"Dialogue agents for artificial intelligence-based conversational systems for cognitively disabled: A systematic review[J]","author":"Huq","year":"2022","journal-title":"Disability and Rehabilitation: Assistive Technology"},{"key":"10.3233\/JIFS-233029_ref15","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1162\/tacl_a_00300","article-title":"Spanbert: Improving pre-training by representing and predicting spans[J]","volume":"8","author":"Joshi","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"10.3233\/JIFS-233029_ref16","doi-asserted-by":"crossref","unstructured":"Ko M. , Lee J. , Kim H. et al., Look at the First Sentence: Position Bias in Question Answering[C], Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020:1109\u20131121.","DOI":"10.18653\/v1\/2020.emnlp-main.84"},{"key":"10.3233\/JIFS-233029_ref17","doi-asserted-by":"crossref","unstructured":"Lai G. , Xie Q. , Liu H. et al., Race: Large-scale reading comprehension dataset from examinations[J], arXiv preprint arXiv:1704.04683, 2017.","DOI":"10.18653\/v1\/D17-1082"},{"key":"10.3233\/JIFS-233029_ref18","first-page":"989","article-title":"Why Machine Reading Comprehension Models Learn Shortcuts?[C]","author":"Lai","year":"2021","journal-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021"},{"key":"10.3233\/JIFS-233029_ref19","doi-asserted-by":"crossref","unstructured":"Liu W. , Zhou P. , Zhao Z. et al., K-bert: Enabling language representation with knowledge graph[C], Proceedings of the AAAI Conference on Artificial Intelligence 34(03) (2020), 2901\u20132908.","DOI":"10.1609\/aaai.v34i03.5681"},{"issue":"1","key":"10.3233\/JIFS-233029_ref20","doi-asserted-by":"crossref","first-page":"113","DOI":"10.3390\/electronics12010113","article-title":"SHO-CNN: A metaheuristic optimization of a convolutional neural network for multi-label news classification[J]","volume":"12","author":"Nadeem","year":"2022","journal-title":"Electronics"},{"key":"10.3233\/JIFS-233029_ref21","first-page":"16292","article-title":"Introspective distillation for robust question answering[J]","volume":"34","author":"Niu","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10.3233\/JIFS-233029_ref22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2020\/8839524","article-title":"Text messaging-based medical diagnosis using natural language processing and fuzzy logic[J]","volume":"2020","author":"Omoregbe","year":"2020","journal-title":"Journal of Healthcare Engineering"},{"key":"10.3233\/JIFS-233029_ref23","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1016\/j.neucom.2022.04.020","article-title":"Adversarial attack and defense technologies in natural language processing: A survey[J]","volume":"492","author":"Qiu","year":"2022","journal-title":"Neurocomputing"},{"key":"10.3233\/JIFS-233029_ref24","first-page":"784","article-title":"Know What You Don\u2019t Know: Unanswerable Questions for SQuAD[C]","author":"Rajpurkar","year":"2018","journal-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics"},{"key":"10.3233\/JIFS-233029_ref25","doi-asserted-by":"crossref","unstructured":"Rajpurkar P. , Zhang J. , Lopyrev K. et al., Squad: 100,000+ questions for machine comprehension of text[J], arXiv preprint arXiv:1606.05250, 2016.","DOI":"10.18653\/v1\/D16-1264"},{"key":"10.3233\/JIFS-233029_ref26","unstructured":"Seo M. , Kembhavi A. , Farhadi A. et al., Bidirectional attention flow for machine comprehension[J], arXiv preprint arXiv:1611.01603, 2016."},{"key":"10.3233\/JIFS-233029_ref27","doi-asserted-by":"crossref","unstructured":"Sugawara S. , Inui K. , Sekine S. et al., What Makes Reading Comprehension Questions Easier?[C], Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018:4208\u20134219.","DOI":"10.18653\/v1\/D18-1453"},{"key":"10.3233\/JIFS-233029_ref28","first-page":"30","article-title":"Attention is all you need[J]","author":"Vaswani","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10.3233\/JIFS-233029_ref29","first-page":"28","article-title":"Pointer networks[J]","author":"Vinyals","year":"2015","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10.3233\/JIFS-233029_ref30","unstructured":"Wang S. and Jiang J. , Machine comprehension using match-lstm and answer pointer[J], arXiv preprint arXiv:1608.07905, 2016."},{"key":"10.3233\/JIFS-233029_ref31","doi-asserted-by":"crossref","first-page":"189","DOI":"10.18653\/v1\/P17-1018","article-title":"Gated self-matching networks for reading comprehension and question answering","author":"Wang","year":"2017","journal-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics"},{"key":"10.3233\/JIFS-233029_ref32","first-page":"10890","article-title":"R-drop: Regularized dropout for neural networks[J]","volume":"34","author":"Wu","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10.3233\/JIFS-233029_ref33","first-page":"32","article-title":"Xlnet: Generalized autoregressive pretraining for language understanding[J]","author":"Yang","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10.3233\/JIFS-233029_ref34","unstructured":"Yu A.W. , Dohan D. , Luong M.T. et al., QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension[C], International Conference on Learning Representations."},{"key":"10.3233\/JIFS-233029_ref35","unstructured":"Zhu B. , Wang T. and Kong L. , Counterfactual QA: Eliminating Bias in Question Answering, (2021)."}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JIFS-233029","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:42:01Z","timestamp":1777455721000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JIFS-233029"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,4]]},"references-count":35,"journal-issue":{"issue":"5"},"URL":"https:\/\/doi.org\/10.3233\/jifs-233029","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,4]]}}}