{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T09:49:42Z","timestamp":1763027382910,"version":"3.45.0"},"reference-count":55,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T00:00:00Z","timestamp":1762992000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Project of Basic Science (Natural Science) Research in Jiangsu Province","award":["25KJA413002"],"award-info":[{"award-number":["25KJA413002"]}]},{"name":"KTH Royal Institute of Technology with the industrial research project ADinSOS","award":["2019065006"],"award-info":[{"award-number":["2019065006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>Modern companies often rely on integrating an extensive network of suppliers to organize and produce industrial artifacts. Within this process, it is critical to maintain sustainability and flexibility by analyzing and managing information from the supply chain. In particular, there is a continuous demand to automatically analyze and infer information from extensive datasets structured in various forms, such as natural language and domain-specific models. The advancement of Large Language Models (LLM) presents a promising solution to address this challenge. By leveraging prompts that contain the necessary information provided by humans, LLM can generate insightful responses through analysis and reasoning over the provided content. However, the quality of these responses is still affected by the inherent opaqueness of LLM, stemming from their complex architectures, thus weakening their trustworthiness and limiting their applicability across different fields. To address this issue, this work presents a framework to leverage the graph-based LLM to support the analysis of supply chain information by combining the LLM and domain knowledge. Specifically, this work proposes an integration of LLM and domain knowledge to support an analysis of the supply chain as follows: (1) constructing a graph-based knowledge base to describe and model the domain knowledge; (2) creating prompts to support the retrieval of the graph-based models and guide the generation of LLM; (3) generating responses via LLM to support the analysis and reason about information across the supply chain. We demonstrate the proposed framework in the tasks of entity classification, link prediction, and reasoning across entities. Compared to the average performance of the best methods in the comparative studies, the proposed framework achieves a significant improvement of 59%, increasing the ROUGE-1 F1 score from 0.42 to 0.67.<\/jats:p>","DOI":"10.3390\/informatics12040124","type":"journal-article","created":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T09:10:45Z","timestamp":1763025045000},"page":"124","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Leveraging the Graph-Based LLM to Support the Analysis of Supply Chain Information"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8028-3607","authenticated-orcid":false,"given":"Peng","family":"Su","sequence":"first","affiliation":[{"name":"School of Automation, Nanjing Institute of Technology, Nanjing 211167, China"},{"name":"Department of Engineering Design, KTH Royal Institute of Technology, 11428 Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0038-9234","authenticated-orcid":false,"given":"Rui","family":"Xu","sequence":"additional","affiliation":[{"name":"Department of Engineering Design, KTH Royal Institute of Technology, 11428 Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7048-0108","authenticated-orcid":false,"given":"Dejiu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Engineering Design, KTH Royal Institute of Technology, 11428 Stockholm, Sweden"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1080\/00207543.2017.1387680","article-title":"Ripple effect in the supply chain: An analysis and recent literature","volume":"56","author":"Dolgui","year":"2018","journal-title":"Int. J. Prod. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1080\/17517575.2020.1762004","article-title":"Risk assessment of supply-chain systems: A probabilistic inference method","volume":"14","author":"Zheng","year":"2020","journal-title":"Enterp. Inf. Syst."},{"key":"ref_3","first-page":"187","article-title":"Industry 4.0 in logistics and supply chain management: A systematic literature review","volume":"33","author":"Abdirad","year":"2021","journal-title":"Eng. Manag. J."},{"key":"ref_4","first-page":"100257","article-title":"Industry 5.0: A survey on enabling technologies and potential applications","volume":"26","author":"Maddikunta","year":"2022","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_5","first-page":"100874","article-title":"Designing a knowledge-enhanced framework to support supply chain information management","volume":"47","author":"Su","year":"2025","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1007\/s10479-016-2236-y","article-title":"Big Data and supply chain management: A review and bibliometric analysis","volume":"270","author":"Mishra","year":"2018","journal-title":"Ann. Oper. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1080\/17517575.2013.879209","article-title":"Using integrated information systems in supply chain management","volume":"9","author":"Varajao","year":"2015","journal-title":"Enterp. Inf. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"7537","DOI":"10.1109\/TII.2022.3146598","article-title":"Industrial IoT and long short-term memory network-enabled genetic indoor-tracking for factory logistics","volume":"18","author":"Wu","year":"2022","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Singla, T., Anandayuvaraj, D., Kalu, K.G., Schorlemmer, T.R., and Davis, J.C. (2023, January 30). An empirical study on using large language models to analyze software supply chain security failures. Proceedings of the 2023 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses, Copenhagen, Denmark.","DOI":"10.1145\/3605770.3625214"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","volume":"55","author":"Ji","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_11","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"Volume 35","author":"Wei","year":"2022","journal-title":"Advances in Neural Information Processing Systems, Proceedings of the Conference and Workshop on Neural Information Processing Systems 2022, New Orleans, LA, USA, 27 November\u20138 December 2022"},{"key":"ref_12","unstructured":"Peng, B., Zhu, Y., Liu, Y., Bo, X., Shi, H., Hong, C., Zhang, Y., and Tang, S. (2024). Graph retrieval-augmented generation: A survey. arXiv."},{"key":"ref_13","first-page":"100566","article-title":"IDS-KG: An industrial dataspace-based knowledge graph construction approach for smart maintenance","volume":"38","author":"Wang","year":"2024","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Su, P., Xu, R., Wu, W., and Chen, D. (2025). Integrating Large Language Model and Logic Programming for Tracing Renewable Energy Use Across Supply Chain Networks. Appl. Syst. Innov., 8.","DOI":"10.3390\/asi8060160"},{"key":"ref_15","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive nlp tasks","volume":"Volume 33","author":"Lewis","year":"2020","journal-title":"Advances in Neural Information Processing Systems, Proceedings of the Conference and Workshop on Neural Information Processing Systems 2020, Vancouver, BC, Canada, 6\u201312 December 2020"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1109\/JAS.2023.123618","article-title":"A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development","volume":"10","author":"Wu","year":"2023","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yenduri, G., Srivastava, G., Maddikunta, P.K.R., Jhaveri, R.H., Wang, W., Vasilakos, A.V., and Gadekallu, T.R. (2023). Generative pre-trained transformer: A comprehensive review on enabling technologies, potential applications, emerging challenges, and future directions. arXiv.","DOI":"10.1109\/ACCESS.2024.3389497"},{"key":"ref_18","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA."},{"key":"ref_19","unstructured":"Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2025, November 04). Improving Language Understanding by Generative Pre-Training. Available online: https:\/\/www.mikecaptain.com\/resources\/pdf\/GPT-1.pdf."},{"key":"ref_20","unstructured":"Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., and Le, Q.V. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems, Proceedings of the Conference and Workshop on Neural Information Processing Systems 2019, Vancouver, BC, Canada, 8\u201314 December 2019, NeurIPS Foundation."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sahoo, P., Singh, A.K., Saha, S., Jain, V., Mondal, S., and Chadha, A. (2024). A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv.","DOI":"10.1007\/979-8-8688-0569-1_4"},{"key":"ref_22","unstructured":"Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., and Liu, T. (2022). A survey on in-context learning. arXiv."},{"key":"ref_23","unstructured":"Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E.H., Sch\u00e4rli, N., and Zhou, D. (2023, January 23\u201329). Large language models can be easily distracted by irrelevant context. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA."},{"key":"ref_24","unstructured":"Hu, H., Lu, H., Zhang, H., Song, Y.Z., Lam, W., and Zhang, Y. (2023). Chain-of-symbol prompting elicits planning in large langauge models. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Su, P., and Chen, D. (2024). Adopting graph neural networks to analyze human\u2013object interactions for inferring activities of daily living. Sensors, 24.","DOI":"10.3390\/s24082567"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"8622","DOI":"10.1109\/TKDE.2024.3469578","article-title":"Large language models on graphs: A comprehensive survey","volume":"26","author":"Jin","year":"2024","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_27","unstructured":"Guo, T., Guo, K., Liang, Z., Guo, Z., Chawla, N.V., Wiest, O., and Zhang, X. (2023). What indeed can GPT models do in chemistry? A comprehensive benchmark on eight tasks. arXiv."},{"key":"ref_28","unstructured":"He, X., Bresson, X., Laurent, T., Perold, A., LeCun, Y., and Hooi, B. (2023). Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation learning. arXiv."},{"key":"ref_29","unstructured":"Lacombe, R., Gaut, A., He, J., L\u00fcdeke, D., and Pistunova, K. (2023). Extracting molecular properties from natural language with multimodal contrastive learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kim, J., Kwon, Y., Jo, Y., and Choi, E. (2023). KG-GPT: A general framework for reasoning on knowledge graphs using large language models. arXiv.","DOI":"10.18653\/v1\/2023.findings-emnlp.631"},{"key":"ref_31","unstructured":"Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., and Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv."},{"key":"ref_32","unstructured":"Chen, J., Lin, H., Han, X., and Sun, L. (2024, January 20\u201327). Benchmarking large language models in retrieval-augmented generation. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Huang, Y., Li, Y., Xu, Y., Zhang, L., Gan, R., Zhang, J., and Wang, L. (2023, January 9\u201314). Mvp-tuning: Multi-view knowledge retrieval with prompt tuning for commonsense reasoning. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, ON, Canada.","DOI":"10.18653\/v1\/2023.acl-long.750"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mavromatis, C., and Karypis, G. (2024). Gnn-rag: Graph neural retrieval for large language model reasoning. arXiv.","DOI":"10.18653\/v1\/2025.findings-acl.856"},{"key":"ref_35","unstructured":"Li, B., Mellou, K., Zhang, B., Pathuri, J., and Menache, I. (2023). Large language models for supply chain optimization. arXiv."},{"key":"ref_36","first-page":"1877","article-title":"Language models are few-shot learners","volume":"Volume 33","author":"Brown","year":"2020","journal-title":"Advances in Neural Information Processing Systems, Proceedings of the Conference and Workshop on Neural Information Processing Systems 2020, Vancouver, BC, Canada, 6\u201312 December 2020"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2951","DOI":"10.1016\/j.ifacol.2025.09.496","article-title":"LLMs in Supply Chain Management: Opportunities and Case Study","volume":"59","author":"Zheng","year":"2025","journal-title":"IFAC-PapersOnLine"},{"key":"ref_38","unstructured":"Aghaei, R., Kiaei, A.A., Boush, M., Vahidi, J., Barzegar, Z., and Rofoosheh, M. (2025). The Potential of Large Language Models in Supply Chain Management: Advancing Decision-Making, Efficiency, and Innovation. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"6","DOI":"10.26794\/2308-944X-2024-12-3-6-27","article-title":"A systematic review on exploring the influence of Industry 4.0 technologies to enhance supply chain visibility and operational efficiency","volume":"12","author":"Khan","year":"2024","journal-title":"Rev. Bus. Econ. Stud."},{"key":"ref_40","unstructured":"Han, K. (2024). Applying graph neural network to SupplyGraph for supply chain network. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1080\/17517575.2011.593104","article-title":"An approach for formalising the supply chain operations","volume":"5","author":"Panetto","year":"2011","journal-title":"Enterp. Inf. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"5596","DOI":"10.1080\/00207543.2022.2100841","article-title":"Towards knowledge graph reasoning for supply chain risk management using graph neural networks","volume":"62","author":"Kosasih","year":"2024","journal-title":"Int. J. Prod. Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"103627","DOI":"10.1016\/j.artint.2021.103627","article-title":"Knowledge graphs as tools for explainable machine learning: A survey","volume":"302","author":"Tiddi","year":"2022","journal-title":"Artif. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Liu, S., Zeng, Z., Chen, L., Ainihaer, A., Ramasami, A., Chen, S., Xu, Y., Wu, M., and Wang, J. (2025). TigerVector: Supporting Vector Search in Graph Databases for Advanced RAGs. arXiv.","DOI":"10.1145\/3722212.3724456"},{"key":"ref_45","first-page":"132876","article-title":"G-retriever: Retrieval-augmented generation for textual graph understanding and question answering","volume":"Volume 37","author":"He","year":"2024","journal-title":"Advances in Neural Information Processing Systems, Proceedings of the Conference and Workshop on Neural Information Processing Systems 2024, Vancouver, BC, Canada, 10\u201315 December 2024"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Shu, Y., Yu, Z., Li, Y., Karlsson, B.F., Ma, T., Qu, Y., and Lin, C.Y. (2022). Tiara: Multi-grained retrieval for robust question answering over large knowledge bases. arXiv.","DOI":"10.18653\/v1\/2022.emnlp-main.555"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1075\/ijcl.15.4.02lu","article-title":"Automatic analysis of syntactic complexity in second language writing","volume":"15","author":"Lu","year":"2010","journal-title":"Int. J. Corpus Linguist."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"41805","DOI":"10.1109\/ACCESS.2020.2977136","article-title":"KnowIME: A system to construct a knowledge graph for intelligent manufacturing equipment","volume":"8","author":"Yan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1016\/j.jmsy.2024.03.010","article-title":"Manufacturing service capability prediction with Graph Neural Networks","volume":"74","author":"Li","year":"2024","journal-title":"J. Manuf. Syst."},{"key":"ref_50","unstructured":"Team, G., Anil, R., Borgeaud, S., Alayrac, J.B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., Hauth, A., and Millican, K. (2023). Gemini: A family of highly capable multimodal models. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Es, S., James, J., Anke, L.E., and Schockaert, S. (2024, January 17\u201322). Ragas: Automated evaluation of retrieval augmented generation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, St. Julian\u2019s, Malta.","DOI":"10.18653\/v1\/2024.eacl-demo.16"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.ifacol.2016.03.014","article-title":"Modeling and random path-based direct encoding for a closed loop supply chain model with flexible delivery paths","volume":"49","author":"Behmanesh","year":"2016","journal-title":"IFAC-PapersOnLine"},{"key":"ref_53","unstructured":"Zhang, J. (2023). Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Bast, H., and Haussmann, E. (2015, January 19\u201323). More accurate question answering on freebase. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Australia.","DOI":"10.1145\/2806416.2806472"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1109\/OJITS.2025.3563428","article-title":"Adopting Graph Neural Networks to Understand and Reason about Dynamic Driving Scenarios","volume":"6","author":"Su","year":"2025","journal-title":"IEEE Open J. Intell. Transp. Syst."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/124\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T09:46:16Z","timestamp":1763027176000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/124"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,13]]},"references-count":55,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["informatics12040124"],"URL":"https:\/\/doi.org\/10.3390\/informatics12040124","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,13]]}}}