{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,5]],"date-time":"2026-08-05T18:39:25Z","timestamp":1785955165372,"version":"3.56.0"},"reference-count":19,"publisher":"Springer Science and Business Media LLC","issue":"19","license":[{"start":{"date-parts":[[2024,7,6]],"date-time":"2024-07-06T00:00:00Z","timestamp":1720224000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2024,7,6]],"date-time":"2024-07-06T00:00:00Z","timestamp":1720224000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2024,10]]},"DOI":"10.1007\/s10489-024-05453-7","type":"journal-article","created":{"date-parts":[[2024,7,6]],"date-time":"2024-07-06T06:02:00Z","timestamp":1720245720000},"page":"8902-8923","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Deep context transformer: bridging efficiency and contextual understanding of transformer models"],"prefix":"10.1007","volume":"54","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9050-2984","authenticated-orcid":false,"given":"Shadi","family":"Ghaith","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,7,6]]},"reference":[{"key":"5453_CR1","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, pp 4171\u20134186"},{"key":"5453_CR2","unstructured":"Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et\u00a0al (2020) Language models are few-shot learners. In: Advances in neural information processing systems, volume\u00a033. NeurIPS"},{"issue":"4","key":"5453_CR3","doi-asserted-by":"publisher","first-page":"242","DOI":"10.3390\/info14040242","volume":"14","author":"N Patwardhan","year":"2023","unstructured":"Patwardhan N, Marrone S, Sansone C (2023) Transformers in the real world: A survey on nlp applications. Information 14(4):242","journal-title":"Information"},{"key":"5453_CR4","doi-asserted-by":"publisher","first-page":"2011","DOI":"10.1007\/s11431-020-1692-3","volume":"63","author":"C Zhang","year":"2020","unstructured":"Zhang C, Li Y, Du N, Fan W, Yu P (2020) Challenges in building intelligent open-domain dialog systems. Sci China Technol Sci 63:2011\u20132027","journal-title":"Sci China Technol Sci"},{"key":"5453_CR5","doi-asserted-by":"crossref","unstructured":"Feng Z, Guo D, Tang D, Duan N, Feng X, Gong M, Shou L, Qin B, Liu T, Jiang D, Zhou M (2020) Codebert: A pre-trained model for programming and natural languages. In: Findings of the association for computational linguistics: EMNLP 2020, Online, November 2020. Association for Computational Linguistics, pp 1536\u20131547","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"5453_CR6","first-page":"123","volume":"45","author":"J Smith","year":"2023","unstructured":"Smith J, Doe J (2023) Chunked transformer: Enhancing efficiency in transformer models. J AI Res 45:123\u2013145","journal-title":"J AI Res"},{"issue":"2","key":"5453_CR7","first-page":"234","volume":"7","author":"K Lee","year":"2023","unstructured":"Lee K, Kim H (2023) Improving transformer models with modified attention mechanisms. Adv Comput Intell 7(2):234\u2013250","journal-title":"Adv Comput Intell"},{"key":"5453_CR8","unstructured":"Kim S, Galley M, Gunasekara C, Lee S, Atkinson A, Peng B, Schulz H, Gao J, Li J, Adada M et al (2020) The eighth dialog system technology challenge. In Proceedings of the AAAI conference on artificial intelligence 34:7782\u20137789"},{"key":"5453_CR9","unstructured":"DSTC8 Organizers (2020) Schema-guided dialogue dataset from dstc8. https:\/\/huggingface.co\/datasets\/schema_guided_dstc8"},{"key":"5453_CR10","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s10462-018-9662-y","volume":"53","author":"J Deriu","year":"2020","unstructured":"Deriu J, Rodrigo A, Otegi A, Echegoyen G, Rosset S, Agirre E, Cieliebak M (2020) Survey on evaluation methods for dialogue systems. Artif Intell Rev 53:407\u2013442","journal-title":"Artif Intell Rev"},{"key":"5453_CR11","unstructured":"Puri R, Kung D, Janssen G, Zhang W, Domeniconi G, Zolotov V, Dolby J, Chen J, Choudhury M, Decker L, Thost V, Buratti L, Pujar S, Finkler U (2021) Project codenet: A large-scale ai for code dataset for learning a diversity of coding tasks. IBM Res"},{"issue":"15","key":"5453_CR12","first-page":"10041","volume":"33","author":"X Wang","year":"2021","unstructured":"Wang X, Jiang J, Sun Z (2021) Application of the bert-based architecture in fake news detection. Neural Comput Appl 33(15):10041\u201310050","journal-title":"Neural Comput Appl"},{"issue":"1","key":"5453_CR13","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1186\/s40537-022-00590-7","volume":"9","author":"W Wongso","year":"2022","unstructured":"Wongso W, Lucky H, Suhartono D (2022) Pre-trained transformer-based language models for sundanese. J Big Data 9(1):39","journal-title":"J Big Data"},{"key":"5453_CR14","doi-asserted-by":"crossref","unstructured":"Qin G, Feng Y, Van\u00a0Durme B (2023) The NLP task effectiveness of long-range transformers. In: Proceedings of the 17th conference of the european chapter of the association for computational linguistics (EACL), Dubrovnik, Croatia. Association for Computational Linguistics, pp 3774\u20133790","DOI":"10.18653\/v1\/2023.eacl-main.273"},{"issue":"9","key":"5453_CR15","first-page":"1","volume":"54","author":"Y Tay","year":"2022","unstructured":"Tay Y, Dehghani M, Bahri D, Metzler D (2022) Efficient transformers: A survey. ACM Comput Surv 54(9):1\u201335","journal-title":"ACM Comput Surv"},{"key":"5453_CR16","doi-asserted-by":"crossref","unstructured":"Chernyavskiy A, Ilvovsky D, Nakov P (2021) Transformers: \u201cthe end of history\u201d for natural language processing? In: Proceedings of the joint european conference on machine learning and knowledge discovery in databases (ECML PKDD). Springer","DOI":"10.1007\/978-3-030-86523-8_41"},{"issue":"1","key":"5453_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-022-24787-1","volume":"12","author":"Y Yang","year":"2022","unstructured":"Yang Y, Cao J, Wen Y, Zhang P (2022) Multiturn dialogue generation by modeling sentence-level and discourse-level contexts. Sci Report 12(1):1\u201312","journal-title":"Sci Report"},{"key":"5453_CR18","doi-asserted-by":"crossref","unstructured":"Tuteja M, Gonz\u00e1lez\u00a0Jucl\u00e0 D (2023) Long text classification using transformers with paragraph selection strategies. In: Proceedings of the natural legal language processing workshop 2023, Singapore, December. Association for Computational Linguistics, pp 17\u201324","DOI":"10.18653\/v1\/2023.nllp-1.3"},{"key":"5453_CR19","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems 5998\u20136008"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-024-05453-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-024-05453-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-024-05453-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,15]],"date-time":"2024-08-15T13:06:52Z","timestamp":1723727212000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-024-05453-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,6]]},"references-count":19,"journal-issue":{"issue":"19","published-print":{"date-parts":[[2024,10]]}},"alternative-id":["5453"],"URL":"https:\/\/doi.org\/10.1007\/s10489-024-05453-7","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"value":"0924-669X","type":"print"},{"value":"1573-7497","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,6]]},"assertion":[{"value":"6 April 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 July 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that there are no competing interests regarding the publication of this paper. This research was conducted independently and did not receive specific funding from any agencies in the public, commercial, or not-for-profit sectors. The development and testing of the Deep Context Transformer model was performed without any influence from software or hardware vendors, ensuring an unbiased approach in the research. Any software or tools mentioned in this paper are cited purely based on their relevance and utility to the research, and do not imply endorsement.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"The research presented in this paper utilized the Schema-Guided Dialogue dataset from the 8th Dialogue System Technology Challenge (DSTC8), hosted on the Hugging Face platform (). This dataset is publicly available and was designed for research purposes, ensuring that all data used in this study adheres to ethical standards for data use in research. No ethical approval was required as the dataset contains no personal or sensitive information, and it is specifically intended for public use in dialogue system research. The dataset was used in accordance with the terms and conditions provided by the hosting platform.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval and informed consent"}}]}}