{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T17:04:02Z","timestamp":1768323842397,"version":"3.49.0"},"reference-count":53,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T00:00:00Z","timestamp":1768262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Zhejiang Key R&D Plan","award":["2017C03047"],"award-info":[{"award-number":["2017C03047"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Retrieval-Augmented Generation (RAG) is widely used for long-text summarization due to its efficiency and scalability. However, standard RAG methods flatten documents into independent chunks, disrupting sequential flow and thematic structure, resulting in significant loss of contextual information. This paper presents MOEGAT, a novel graph-enhanced retrieval framework that addresses this limitation by explicitly modeling document structure. MOEGAT constructs an Orthogonal Context Graph to capture sequential discourse and global semantic relationships\u2014long-range dependencies between non-adjacent text spans that reflect topical similarity and logical associations beyond local context. It then employs a query-aware Mixture-of-Experts Graph Attention Network to dynamically activate specialized reasoning pathways. Experiments conducted on three public long-text summarization datasets demonstrate that MOEGAT achieves state-of-the-art performance. Notably, on the WCEP dataset, it outperforms the previous state-of-the-art Graph of Records (GOR) baseline by 14.9%, 18.1%, and 18.4% on ROUGE-L, ROUGE-1, and ROUGE-2, respectively. These substantial gains, especially the 14.9% improvement in ROUGE-L, reflect significantly better capture of long-range coherence and thematic integrity in summaries. Ablation studies confirm the effectiveness of the orthogonal graph and Mixture-of-Experts components. Overall, this work introduces a novel structure-aware approach to RAG that explicitly models and leverages document structure through an orthogonal graph representation and query-aware Mixture-of-Experts reasoning.<\/jats:p>","DOI":"10.3390\/bdcc10010031","type":"journal-article","created":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T10:13:51Z","timestamp":1768299231000},"page":"31","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["TRACE: Topical Reasoning with Adaptive Contextual Experts"],"prefix":"10.3390","volume":"10","author":[{"given":"Jiabin","family":"Ye","sequence":"first","affiliation":[{"name":"School of Information Engineering, Huzhou University, Huzhou 313000, China"}]},{"given":"Qiuyi","family":"Xin","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Huzhou University, Huzhou 313000, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6760-3154","authenticated-orcid":false,"given":"Chu","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Huzhou University, Huzhou 313000, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1927-7160","authenticated-orcid":false,"given":"Hengnian","family":"Qi","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Huzhou University, Huzhou 313000, China"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/3641289","article-title":"A Survey on Evaluation of Large Language Models","volume":"15","author":"Chang","year":"2024","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3159","DOI":"10.1007\/s11192-020-03630-2","article-title":"Automatic related work section generation: Experiments in scientific document abstracting","volume":"125","author":"Saggion","year":"2020","journal-title":"Scientometrics"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"124456","DOI":"10.1016\/j.eswa.2024.124456","article-title":"Factual Consistency Evaluation of Summarization in the Era of Large Language Models","volume":"254","author":"Luo","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_4","unstructured":"Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. (2024, January 7\u201311). Self-rag: Learning to retrieve, generate, and critique through self-reflection. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ratner, N., Levine, Y., Belinkov, Y., Ram, O., Magar, I., Abend, O., Karpas, E., Shashua, A., Leyton-Brown, K., and Shoham, Y. (2023, January 9\u201314). Parallel Context Windows for Large Language Models. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, ON, Canada.","DOI":"10.18653\/v1\/2023.acl-long.352"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Jiang, Z., Xu, F.F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., and Neubig, G. (2023, January 6\u201310). Active retrieval augmented generation. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Sentosa, Singapore.","DOI":"10.18653\/v1\/2023.emnlp-main.495"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., and Yih, W.-t. (2024, January 16\u201321). Replug: Retrieval-augmented black-box language models. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Washington, DC, USA.","DOI":"10.18653\/v1\/2024.naacl-long.463"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1162\/tacl_a_00638","article-title":"Lost in the Middle: How Language Models Use Long Contexts","volume":"12","author":"Liu","year":"2024","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_9","unstructured":"Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R.O., and Larson, J. (2024). From local to global: A graph rag approach to query-focused summarization. arXiv."},{"key":"ref_10","unstructured":"Procko, T.T., and Ochoa, O. (October, January 30). Graph Retrieval-Augmented Generation for Large Language Models: A Survey. Proceedings of the Conference on AI, Science, Engineering, and Technology (AIxSET), Laguna Hills, CA, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3777378","article-title":"Graph Retrieval-Augmented Generation: A Survey","volume":"44","author":"Peng","year":"2024","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"103311","DOI":"10.1016\/j.artmed.2025.103311","article-title":"MedSumGraph: Enhancing GraphRAG for Medical QA with Summarization and Optimized Prompts","volume":"172","author":"Kim","year":"2025","journal-title":"Artif. Intell. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"AIoa2300068","DOI":"10.1056\/AIoa2300068","article-title":"Almanac\u2014Retrieval-Augmented Language Models for Clinical Medicine","volume":"1","author":"Zakka","year":"2024","journal-title":"NEJM AI"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Borchmann, L., Wisniewski, D., Gretkowski, A., Kosmala, I., Jurkiewicz, D., Szalkiewicz, L., Palka, G., Kaczmarek, K., Kaliska, A., and Gralinski, F. (2020, January 16\u201320). Contract Discovery: Dataset and a Few-Shot Semantic Retrieval Challenge with Competitive Baselines. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Online.","DOI":"10.18653\/v1\/2020.findings-emnlp.380"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gubanov, M., Pyayt, A., and Karolak, A. (2024, January 21\u201325). Cancerkg. org-a web-scale, interactive, verifiable knowledge graph-llm hybrid for assisting with optimal cancer treatment and care. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, Boise, ID, USA.","DOI":"10.1145\/3627673.3680094"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Sarmah, B., Mehta, D., Hall, B., Rao, R., Patel, S., and Pasquali, S. (2024, January 14\u201317). Hybridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. Proceedings of the 5th ACM International Conference on AI in Finance, Brooklyn, NY, USA.","DOI":"10.1145\/3677052.3698671"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hu, Y., Lei, Z., Zhang, Z., Pan, B., Ling, C., and Zhao, L. (2025). GRAG: Graph Retrieval-Augmented Generation. Human Language Technology\u2014The Baltic Perspectiv, IOS Press.","DOI":"10.18653\/v1\/2025.findings-naacl.232"},{"key":"ref_18","unstructured":"Zhao, Y., Zhu, J., Guo, Y., He, K., and Li, X. (2025). E^ 2GraphRAG: Streamlining Graph-based RAG for High Efficiency and Effectiveness. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wu, W., Li, W., Xiao, X., Liu, J., Cao, Z., Li, S., Wu, H., and Wang, H. (2021, January 1\u20136). BASS: Boosting Abstractive Summarization with Unified Semantic Graph. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Online.","DOI":"10.18653\/v1\/2021.acl-long.472"},{"key":"ref_20","unstructured":"Zhang, H., Feng, T., and You, J. (August, January 27). Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Yu, B., Shu, X., Mengge, X., Liu, T., and Guo, L. (2021, January 1\u20136). From what to why: Improving relation extraction with rationale graph. Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online.","DOI":"10.18653\/v1\/2021.findings-acl.8"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"118110","DOI":"10.1016\/j.eswa.2022.118110","article-title":"KEST: A graph-based keyphrase extraction technique for tweets summarization using Markov decision process","volume":"209","author":"Garg","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_23","first-page":"4800","article-title":"Hierarchical Graph Representation Learning with Differentiable Pooling","volume":"31","author":"Ying","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhao, C., Zhou, X., Xie, X., and Zhang, Y. (2024). Hierarchical Attention Graph for Scientific Document Summarization in Global and Local Level. Human Language Technology\u2014The Baltic Perspectiv, IOS Press.","DOI":"10.18653\/v1\/2024.findings-naacl.45"},{"key":"ref_25","unstructured":"Liu, Y., and Lapata, M. (August, January 28). Hierarchical Transformers for Multi-Document Summarization. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_26","unstructured":"Zhang, X., Wei, F., and Zhou, M. (28\u20132, January 28). HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_27","unstructured":"Plenz, M., and Frank, A. (2024, January 11\u201316). Graph Language Models. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, T., Chen, X., Du, X., Rashwan, A., Yang, F., Chen, H., Wang, Z., and Li, Y. (2023, January 1\u20136). AdaMV-MoE: Adaptive Multi-Task Vision Mixture-of-Experts. Proceedings of the IEEE International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.01591"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3424","DOI":"10.1109\/TPAMI.2025.3532688","article-title":"Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts","volume":"47","author":"Li","year":"2025","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhang, J., Zhang, X., Yu, J., Tang, J., Tang, J., Li, C., and Chen, H. (2022, January 22\u201327). Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Dublin, Ireland.","DOI":"10.18653\/v1\/2022.acl-long.396"},{"key":"ref_31","unstructured":"Liang, X., Niu, S., Li, Z., Zhang, S., Song, S., Wang, H., Yang, J., Xiong, F., Tang, B., and Xi, C. (2024). Empowering Large Language Models to Set Up a Knowledge Retrieval Indexer Via Self-Learning. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, S., Cao, J., Yang, R., and Wen, Z. (2022). Generating a Structured Summary of Numerous Academic Papers: Dataset and Method. Comput. Res. Repos., 4280\u20134288.","DOI":"10.24963\/ijcai.2022\/591"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gutierrez, B.J., Shu, Y., Gu, Y., Yasunaga, M., and Su, Y. (2024, January 10\u201315). HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. Proceedings of the 38th Conference on Neural Information Processing Systems, Vancouver, BC, Canada.","DOI":"10.52202\/079017-1902"},{"key":"ref_34","unstructured":"He, X., Tian, Y., Sun, Y., Chawla, N.V., Laurent, T., LeCun, Y., Bresson, X., and Hooi, B. (2024, January 10\u201315). G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering. Proceedings of the 38th Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, Z., Guo, Q., Shao, J., Song, L., Bian, J., Zhang, J., and Wang, R. (May, January 29). Graph Neural Network Enhanced Retrieval for Question Answering of Large Language Models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, Albuquerque, NM, USA.","DOI":"10.18653\/v1\/2025.naacl-long.337"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mavromatis, C., and Karypis, G. (August, January 27). GNN-RAG: Graph Neural Retrieval for Efficient Large Language Model Reasoning on Knowledge Graphs. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025, Vienna, Austria.","DOI":"10.18653\/v1\/2025.findings-acl.856"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hu, Y., and Lu, Y. (2024). RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing. arXiv.","DOI":"10.2139\/ssrn.4900122"},{"key":"ref_38","first-page":"100807","article-title":"Knowledge Graph Enhanced Retrieval-Augmented Generation for Failure Mode and Effects Analysis","volume":"45","author":"Bahr","year":"2025","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Suryawanshi, S., Waghmode, S., Sawant, R., and Gupta, M. (2025, January 14\u201316). A Knowledge Graph-based RAG for Cross-Document Information Extraction. Proceedings of the 5th International Conference on Pervasive Computing Social Networking, Salem, India.","DOI":"10.1109\/ICPCSN65854.2025.11035854"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhu, X., Xie, Y., Liu, Y., Li, Y., and Hu, W. (May, January 29). Knowledge Graph-Guided Retrieval Augmented Generation. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, Albuquerque, NM, USA.","DOI":"10.18653\/v1\/2025.naacl-long.449"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhong, M., Yin, D., Yu, T., Zaidi, A., Mutuma, M., Jha, R., Awadallah, A.H., Celikyilmaz, A., Liu, Y., and Qiu, X. (2021, January 6\u201311). QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization. Proceedings of the North American Chapter of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2021.naacl-main.472"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Kry\u015bci\u0144ski, W., Rajani, N., Agarwal, D., Xiong, C., and Radev, D. (2022). BookSum: A Collection of Datasets for Long-form Narrative Summarization. arXiv.","DOI":"10.18653\/v1\/2022.findings-emnlp.488"},{"key":"ref_43","unstructured":"Ghalandari, D.G., Hokamp, C., Pham, N.T., Glover, J., and Ifrim, G. (2020, January 5\u201310). A Large-Scale Multi-Document Summarization Dataset from the Wikipedia Current Events Portal. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Online."},{"key":"ref_44","unstructured":"Lin, C.-Y. (2004, January 25\u201326). ROUGE: A Package for Automatic Evaluation of Summaries. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1561\/1500000019","article-title":"The Probabilistic Relevance Framework: BM25 and Beyond","volume":"3","author":"Robertson","year":"2009","journal-title":"Found. Trends Inf. Retr."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Tawfeeq, N.Z., Abed, W.S., and Ghazal, O.G. (2020, January 24\u201325). A semantic model of morphological information retrieval: A comparative accumulative analysis. Proceedings of the 2020 2nd Annual International Conference on Information and Sciences (AiCIS), Fallujah, Iraq.","DOI":"10.1109\/AiCIS51645.2020.00011"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Reimers, N., and Gurevych, I. (2019, January 3\u20137). Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1410"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Lin, S.-C., Asai, A., Li, M., Oguz, B., Lin, J., Mehdad, Y., Yih, W.-t., and Chen, X. (2023, January 6\u201310). How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore.","DOI":"10.18653\/v1\/2023.findings-emnlp.423"},{"key":"ref_49","first-page":"2835","article-title":"Unsupervised Dense Information Retrieval with Contrastive Learning","volume":"2022","author":"Izacard","year":"2021","journal-title":"Trans. Mach. Learn. Res."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Karpukhin, V., O\u011fuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. (2020, January 16\u201320). Dense passage retrieval for open-domain question answering. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Grover, A., and Leskovec, J. (2016, January 13\u201317). Node2vec: Scalable Feature Learning for Networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201816), New York, NY, USA.","DOI":"10.1145\/2939672.2939754"},{"key":"ref_52","unstructured":"Feng, T., Han, P., Lin, G., Liu, G., and You, J. (2025, November 12). Thought-Retriever: Don\u2019t Just Retrieve Raw Data, Retrieve Thoughts. International Conference on Learning Representations Workshop: How Far Are We from AGI. Available online: https:\/\/openreview.net\/pdf\/006ed5ed24038d996f1ced687940c3a1078d72d0.pdf."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Camacho-Collados, J., and Bollegala, D. (2023, January 6\u201310). A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore.","DOI":"10.18653\/v1\/2023.emnlp-main.683"}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/10\/1\/31\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T10:33:11Z","timestamp":1768300391000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/10\/1\/31"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,13]]},"references-count":53,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["bdcc10010031"],"URL":"https:\/\/doi.org\/10.3390\/bdcc10010031","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,13]]}}}