{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T05:09:29Z","timestamp":1781154569772,"version":"3.54.1"},"reference-count":31,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,1,20]],"date-time":"2023-01-20T00:00:00Z","timestamp":1674172800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62022013"],"award-info":[{"award-number":["62022013"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2021YFB1715000"],"award-info":[{"award-number":["2021YFB1715000"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["62022013"],"award-info":[{"award-number":["62022013"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2021YFB1715000"],"award-info":[{"award-number":["2021YFB1715000"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We construct a spacecraft performance-fault relationship graph of the control system, which can help space robots locate and repair spacecraft faults quickly. In order to improve the performance-fault relationship graph, we improve the Deep Deterministic Policy Gradient (DDPG) algorithm, and propose a relationship prediction method that combines representation learning reasoning with deep reinforcement learning reasoning. We take the spacecraft performance-fault relationship graph as the agent learning environment and adopt reinforcement learning to realize the optimal interaction between the agent and the environment. Meanwhile, our model uses a deep neural network to construct a complex value function and strategy function, which makes the agent have excellent perceptual decision-making ability and accurate value judgment ability. We evaluate our model on a performance-fault relationship graph of the control system. The experimental results show that our model has high prediction speed and accuracy, which can completely infer the optimal relationship path between entities to complete the spacecraft performance-fault relationship graph.<\/jats:p>","DOI":"10.3390\/s23031223","type":"journal-article","created":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T01:36:26Z","timestamp":1674437786000},"page":"1223","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["An Improved DDPG and Its Application in Spacecraft Fault Knowledge Graph"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0539-0936","authenticated-orcid":false,"given":"Xiaoyu","family":"Xing","sequence":"first","affiliation":[{"name":"Beijing Institute of Control Engineering, Beijing 100190, China"},{"name":"Science and Technology on Space Intelligent Control Laboratory, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuyi","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Institute of Control Engineering, Beijing 100190, China"},{"name":"Science and Technology on Space Intelligent Control Laboratory, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenjing","family":"Liu","sequence":"additional","affiliation":[{"name":"Beijing Institute of Control Engineering, Beijing 100190, China"},{"name":"Science and Technology on Space Intelligent Control Laboratory, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,20]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"Knowledge graphs","volume":"54","author":"Hogan","year":"2021","journal-title":"J. ACM Comput. Surv. (CSUR)"},{"key":"ref_2","first-page":"582","article-title":"Knowledge graph construction techniques","volume":"53","author":"Liu","year":"2016","journal-title":"J. Comput. Res. Dev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1360\/SSI-2019-0271","article-title":"A survey on the construction methods and applications of scitech big data knowledge graph","volume":"50","author":"Zhou","year":"2020","journal-title":"Sci. Sin. Inf."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Dong, X., Gabrilovich, E., Heitz, G., Horn, W., Lao, N., Murphy, K., Strohmann, T., Sun, S., and Zhang, W. (2014, January 24\u201327). Knowledge vault: A web-scale approach to probabilistic knowledge fusion. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.","DOI":"10.1145\/2623330.2623623"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/JPROC.2015.2483592","article-title":"A review of relational machine learning for knowledge graphs","volume":"104","author":"Nickel","year":"2015","journal-title":"Proc. IEEE"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2724","DOI":"10.1109\/TKDE.2017.2754499","article-title":"Knowledge Graph Embedding: A Survey of Approaches and Applications","volume":"29","author":"Wang","year":"2017","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_7","unstructured":"Lin, Y., Han, X., Xie, R., Liu, Z., and Sun, M. (2018). Knowledge representation learning: A quantitative review. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"489","DOI":"10.3233\/SW-160218","article-title":"Knowledge graph refinement: A survey of approaches and evaluation methods","volume":"8","author":"Paulheim","year":"2016","journal-title":"Semant. Web"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wu, T., Qi, G., Li, C., and Wang, M. (2018). A Survey of Techniques for Constructing Chinese Knowledge Graphs and Their Applications. Sustainability, 10.","DOI":"10.3390\/su10093245"},{"key":"ref_10","first-page":"r5","article-title":"Translating embeddings for modeling multi-relational data","volume":"26","author":"Bordes","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_11","unstructured":"Lao, N., Mitchell, T., and Cohen, W. (2011, January 27\u201331). Random walk inference and learning in a large scale knowledge base. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, Edinburgh, UK."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lao, N., Minkov, E., and Cohen, W. (2015, January 26\u201331). Learning relational features with backward random walks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Beijing, China.","DOI":"10.3115\/v1\/P15-1065"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xiong, W., Hoang, T., and Wang, W.Y. (2017). Deeppath: A reinforcement learning method for knowledge graph reasoning. arXiv.","DOI":"10.18653\/v1\/D17-1060"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Wang, H., Li, S., Pan, R., and Mao, M. (2019, January 3\u20137). Incorporating graph attention mechanism into knowledge graph reasoning based on deep reinforcement learning. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1264"},{"key":"ref_15","unstructured":"Das, R., Dhuliawala, S., Zaheer, M., Vilnis, L., Durugkar, I., Krishnamurthy, A., Smola, A., and McCallum, A. (2017). Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. arXiv."},{"key":"ref_16","unstructured":"Shen, Y., Chen, J., Huang, P.S., Guo, Y., and Gao, J. (2018). M-walk: Learning to walk over graphs using monte carlo tree search. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zeng, X., He, S., Liu, K., and Zhao, J. (2018, January 2\u20137). Large scaled relation extraction with reinforcement learning. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11950"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lin, X.V., Socher, R., and Xiong, C. (November, January 31). Multi-Hop Knowledge Graph Reasoning with Reward Shaping. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1362"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fu, C., Chen, T., Qu, M., Jin, W., and Ren, X. (2019, January 3\u20137). Collaborative Policy Learning for Open Knowledge Graph Reasoning. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1269"},{"key":"ref_20","unstructured":"Bruna, J., Zaremba, W., Szlam, A., and LeCun, Y. (2013). Spectral networks and locally connected networks on graphs. arXiv."},{"key":"ref_21","unstructured":"Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M. (2018). European Semantic Web Conference, Springer."},{"key":"ref_22","unstructured":"Vashishth, S., Sanyal, S., Nitin, V., and Talukdar, P. (2019). Composition-based multi-relational graph convolutional networks. arXiv."},{"key":"ref_23","unstructured":"Shang, C., Tang, Y., Huang, J., Bi, J., He, X., and Zhou, B. (February, January 27). End-to-end structure-aware convolutional networks for knowledge base completion. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_24","unstructured":"Veli\u010dkovi\u0107, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2017). Graph attention networks. arXiv."},{"key":"ref_25","unstructured":"Nathani, D., Chauhan, J., Sharma, C., and Kaul, M. (August, January 28). Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.neunet.2020.08.021","article-title":"Mgat: Multi-view graph attention networks","volume":"132","author":"Xie","year":"2020","journal-title":"Neural Netw."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chami, I., Wolf, A., Juan, D.C., Sala, F., Ravi, S., and R\u00e9, C. (2020, January 5\u201310). Low-Dimensional Hyperbolic Knowledge Graph Embeddings. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.617"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, D., Cui, P., and Zhu, W. (2016, January 13\u201317). Structural deep network embedding. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939753"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, S., Wei, X., Nogueira dos Santos, C.N., Wang, Z., Nallapati, R., Arnold, A., Xiang, B., Yu, P.S., and Cruz, I.F. (2021, January 19\u201323). Mixed-curvature multi-relational graph neural network for knowledge graph completion. Proceedings of the Web Conference 2021, Ljubljana, Slovenia.","DOI":"10.1145\/3442381.3450118"},{"key":"ref_30","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1002\/wics.101","article-title":"Principal component analysis","volume":"2","author":"Abdi","year":"2010","journal-title":"Wiley Interdiscip. Rev. Comput. Stat."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1223\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:12:21Z","timestamp":1760119941000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1223"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,20]]},"references-count":31,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23031223"],"URL":"https:\/\/doi.org\/10.3390\/s23031223","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,20]]}}}