{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T11:05:13Z","timestamp":1790075113517,"version":"4.0.1"},"reference-count":24,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2026,9,19]],"date-time":"2026-09-19T00:00:00Z","timestamp":1789776000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NUDT Youth Independent Innovation Science Fund","award":["ZK25-20"],"award-info":[{"award-number":["ZK25-20"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Liquid rocket engines (LREs) are mission-critical propulsion systems whose reliability directly affects launch safety and the repeated operation of reusable launch vehicles. Data-driven LRE fault diagnosis remains difficult because labeled fault samples are scarce, engine-to-engine variations cause distribution shifts, and existing evaluations frequently emphasize classification accuracy without controlling for input representation. This study establishes a reliability-oriented benchmark that evaluates seven representative machine-learning and deep-learning methods from four dimensions: diagnostic capability, sensor informativeness, cross-unit generalization, and data efficiency. The experiments use 1106 fault samples from the XJTU-REF dataset, covering nine fault categories, 19 sensor channels, and three simulated engine units. To ensure an apples-to-apples model comparison, all seven methods are first evaluated using the same 19-dimensional steady-state representation. Under this controlled setting, Random Forest achieves the highest accuracy of 89.15% and Macro-F1 score of 88.74%, while the best deep model, 1D-CNN, reaches 86.42%. An additional representation-ablation experiment shows that concatenating the same steady-state features with learned sequence representations increases 1D-CNN accuracy to 87.58%, reducing its gap from Random Forest to 1.57 percentage points. This result demonstrates that the large difference observed under the original heterogeneous-input setting is partly attributable to representation design rather than model family alone. In cross-unit evaluation, Random Forest obtains an average accuracy of 75.37%, whereas Transformer obtains 61.54%, indicating that engine-to-engine variation remains a major deployment challenge. The top five sensors contribute 47.62% of the total Random Forest importance, and, with five labeled samples per fault category, KNN and Random Forest achieve 71.24% and 70.82% accuracy, respectively. These findings provide quantitative guidance for algorithm selection, sensor prioritization, and data-acquisition planning in reusable launch vehicle health-monitoring systems.<\/jats:p>","DOI":"10.3390\/s26185923","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T10:21:48Z","timestamp":1790072508000},"page":"5923","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Beyond Accuracy: A Reliability-Oriented Multi-Dimensional Benchmark of Data-Driven Fault Diagnosis for Liquid Rocket Engines"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7239-1819","authenticated-orcid":false,"given":"Mingyang","family":"Geng","sequence":"first","affiliation":[{"name":"Xichang Satellite Launch Center, Xichang 615000, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"given":"Gang","family":"Zheng","sequence":"additional","affiliation":[{"name":"Xichang Satellite Launch Center, Xichang 615000, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"software","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Long","family":"He","sequence":"additional","affiliation":[{"name":"Xichang Satellite Launch Center, Xichang 615000, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"visualization","vocabulary":"credit"}]},{"given":"Siyu","family":"Zhao","sequence":"additional","affiliation":[{"name":"Xichang Satellite Launch Center, Xichang 615000, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"writing-original-draft","vocabulary":"credit"}]},{"given":"Xiuwei","family":"Yu","sequence":"additional","affiliation":[{"name":"Xichang Satellite Launch Center, Xichang 615000, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"visualization","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,19]]},"reference":[{"key":"ref_1","unstructured":"National Aeronautics and Space Administration (2025). STS-51F. NASA Mission Archive."},{"key":"ref_2","unstructured":"European Space Agency (2005). Ariane 5 ECA Qualification Flights, European Space Agency."},{"key":"ref_3","first-page":"195","article-title":"Anomaly detection configured as a combination of state observer and Mahalanobis-Taguchi method for a rocket engine","volume":"16","author":"Maru","year":"2018","journal-title":"Trans. Jpn. Soc. Aeronaut. Space Sci. Aerosp. Technol. Jpn."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.ifacol.2022.07.112","article-title":"Remaining useful life prediction with uncertainty quantification of liquid propulsion rocket engine combustion chamber","volume":"55","author":"Kanso","year":"2022","journal-title":"IFAC-PapersOnLine"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.actaastro.2018.03.001","article-title":"Fault detection and diagnosis algorithms for an open-cycle liquid propellant rocket engine using the Kalman filter and fault factor methods","volume":"150","author":"Lee","year":"2018","journal-title":"Acta Astronaut."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zeng, Y., Shao, X., Hu, Q., Hu, H., and Pan, H. (2021). Fault diagnosis of liquid rocket engine thrust chamber based on improved augmented particle filter. Proceedings of the 2021 40th Chinese Control Conference (CCC), IEEE.","DOI":"10.23919\/CCC52363.2021.9550493"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1016\/j.actaastro.2020.11.035","article-title":"Data-driven fault detection in a reusable rocket engine using bivariate time-series analysis","volume":"179","author":"Tsutsumi","year":"2021","journal-title":"Acta Astronaut."},{"key":"ref_8","first-page":"6","article-title":"Parity space method for fault diagnosis of liquid rocket engine","volume":"20","author":"Bing","year":"1999","journal-title":"J. Propuls. Technol. Beijing"},{"key":"ref_9","first-page":"74","article-title":"Independent component analysis for fault diagnosis of the liquid rocket engine","volume":"4","author":"Gao","year":"2013","journal-title":"Missiles Space Veh."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Deng, L., Cheng, Y., and Shi, Y. (2022). Fault detection and diagnosis for liquid rocket engines based on long short-term memory and generative adversarial networks. Aerospace, 9.","DOI":"10.3390\/aerospace9080399"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"104552","DOI":"10.1016\/j.engappai.2021.104552","article-title":"Artificial intelligence in prognostics and health management of engineering systems","volume":"108","author":"Ochella","year":"2022","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/j.jspi.2021.04.004","article-title":"A sequential approach to feature selection in high-dimensional additive models","volume":"215","author":"Gong","year":"2021","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3144","DOI":"10.1109\/ACCESS.2019.2961742","article-title":"Steady-state process fault detection for liquid rocket engines based on convolutional auto-encoder and one-class support vector machine","volume":"8","author":"Zhu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"380","DOI":"10.17706\/jsw.14.8.380-387","article-title":"Fault Detection in Liquid-Propellant Rocket Engines Based on Improved PSO-BP Neural Network","volume":"14","author":"Li","year":"2019","journal-title":"J. Softw."},{"key":"ref_15","first-page":"2464","article-title":"Application of QGA-BP for fault detection of liquid rocket engines","volume":"55","author":"Xu","year":"2019","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1016\/j.actaastro.2020.08.019","article-title":"Deep neural network approach for fault detection and diagnosis during startup transient of liquid-propellant rocket engine","volume":"177","author":"Park","year":"2020","journal-title":"Acta Astronaut."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jmsy.2023.02.016","article-title":"Online visual monitoring method for liquid rocket engine nozzle welding based on a multi-task deep learning model","volume":"68","author":"Zhou","year":"2023","journal-title":"J. Manuf. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"108759","DOI":"10.1016\/j.ress.2022.108759","article-title":"A soft-target difference scaling network via relational knowledge distillation for fault detection of liquid rocket engine under multi-source trouble-free samples","volume":"228","author":"Li","year":"2022","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_19","first-page":"1","article-title":"A supervised framework for recognition of liquid rocket engine health state under steady-state process without fault samples","volume":"70","author":"Lv","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_20","first-page":"163","article-title":"An efficient approach for the diagnosis of faults in turbo pump of liquid rocket engine by employing FFT and time-domain features","volume":"16","author":"Aiswarya","year":"2018","journal-title":"Aust. J. Mech. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"03037","DOI":"10.1051\/matecconf\/202030903037","article-title":"Multi-source fault identification based on combined deep learning","volume":"309","author":"Xing","year":"2020","journal-title":"MATEC Web Conf."},{"key":"ref_22","first-page":"1024","article-title":"Sensor failure management in liquid rocket engine using artificial neural network","volume":"79","author":"Flora","year":"2020","journal-title":"J. Sci. Ind. Res."},{"key":"ref_23","first-page":"20","article-title":"Application of Wasserstein distance in fault detection for liquid-propellant rocket engines","volume":"45","author":"Cheng","year":"2023","journal-title":"J. Natl. Univ. Def. Technol."},{"key":"ref_24","first-page":"917","article-title":"Interpretation of the XJTU-REF Liquid Rocket Engine Fault Simulation Dataset","volume":"39","author":"Yuan","year":"2026","journal-title":"Zhendong Gongcheng Xuebao\/J. Vib. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/18\/5923\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T10:23:07Z","timestamp":1790072587000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/18\/5923"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,19]]},"references-count":24,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["s26185923"],"URL":"https:\/\/doi.org\/10.3390\/s26185923","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,19]]}}}