{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T13:00:35Z","timestamp":1763643635538,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,9,30]],"date-time":"2022-09-30T00:00:00Z","timestamp":1664496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"],"award-info":[{"award-number":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"]}]},{"name":"National Natural Science Foundation of China","award":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"],"award-info":[{"award-number":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"]}]},{"name":"the Technology Foundation Strengthening Project Fund","award":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"],"award-info":[{"award-number":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"]}]},{"name":"the ILS Key Laboratory Fund","award":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"],"award-info":[{"award-number":["2018YFB1702401","52275140","2019-JCJQ-JJ-187","WDZC20205250308"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The main gearbox is very important for the operation safety of helicopters, and the oil temperature reflects the health degree of the gearbox; therefore establishing an accurate oil temperature forecasting model is an important step for reliable fault detection. Firstly, in order to achieve accurate gearbox oil temperature forecasting, an improved deep deterministic policy gradient algorithm with a CNN\u2013LSTM basic learner is proposed, which can excavate the complex relationship between oil temperature and working condition. Secondly, a reward incentive function is designed to accelerate the training time costs and to stabilize the model. Further, a variable variance exploration strategy is proposed to enable the agents of the model to fully explore the state space in the early training stage and to gradually converge in the training later stage. Thirdly, a multi-critics network structure is adopted to solve the problem of inaccurate Q-value estimation, which is the key to improving the prediction accuracy of the model. Finally, KDE is introduced to determine the fault threshold to judge whether the residual error is abnormal after EWMA processing. The experimental results show that the proposed model achieves higher prediction accuracy and shorter fault detection time costs.<\/jats:p>","DOI":"10.3390\/e24101394","type":"journal-article","created":{"date-parts":[[2022,10,8]],"date-time":"2022-10-08T04:04:56Z","timestamp":1665201896000},"page":"1394","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["A Fault Detection Method Based on an Oil Temperature Forecasting Model Using an Improved Deep Deterministic Policy Gradient Algorithm in the Helicopter Gearbox"],"prefix":"10.3390","volume":"24","author":[{"given":"Lei","family":"Wei","sequence":"first","affiliation":[{"name":"College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China"},{"name":"Laboratory of Science and Technology on Integrated Logistics Support, National University of Defense Technology, Changsha 410073, China"},{"name":"College of Mechanical and Vehicle Engineering, Hunan University, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1361-1954","authenticated-orcid":false,"given":"Zhe","family":"Cheng","sequence":"additional","affiliation":[{"name":"College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China"},{"name":"Laboratory of Science and Technology on Integrated Logistics Support, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junsheng","family":"Cheng","sequence":"additional","affiliation":[{"name":"College of Mechanical and Vehicle Engineering, Hunan University, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0258-528X","authenticated-orcid":false,"given":"Niaoqing","family":"Hu","sequence":"additional","affiliation":[{"name":"College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China"},{"name":"Laboratory of Science and Technology on Integrated Logistics Support, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3430-557X","authenticated-orcid":false,"given":"Yi","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China"},{"name":"Laboratory of Science and Technology on Integrated Logistics Support, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"111347","DOI":"10.1016\/j.measurement.2022.111347","article-title":"Nonlinear dynamic modeling and analysis of a helicopter planetary gear set for tooth crack diagnosis","volume":"198","author":"Hu","year":"2022","journal-title":"Measurement"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1549","DOI":"10.1016\/j.cja.2019.07.014","article-title":"A multi-criteria fusion feature selection algorithm for fault diagnosis of helicopter planetary gear train","volume":"33","author":"Sun","year":"2020","journal-title":"Chin. J. Aeronaut."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bolvashenkov, I., Kammermann, J., and Herzog, H.G. (2017, January 11\u201314). Electrification of Helicopter: Actual Feasibility and Prospects. Proceedings of the 2017 IEEE Vehicle Power and Propulsion Conference (VPPC), Belfort, France.","DOI":"10.1109\/VPPC.2017.8330883"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kamble, S.B., Desai, V., Jeppu, Y.V. (2015, January 28\u201330). System Identification for Helicopter Longitudinal Dynamics Model\u2014Best Practices. Proceedings of the 2015 International Conference on Industrial Instrumentation and Control (ICIC), Pune, India.","DOI":"10.1109\/IIC.2015.7150793"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Bayoumi, A., Ranson, W., Eisner, L., and Grant, L.E. (2005, January 5\u201312). Cost and effectiveness analysis of the AH-64 and UH-60 on-board vibrations monitoring system. Proceedings of the 2005 IEEE Aerospace Conference, Big Sky, MT, USA.","DOI":"10.1109\/AERO.2005.1559698"},{"key":"ref_6","unstructured":"Song, J.H., Yang, J.W., Rim, M.S., Kim, Y.Y., Kim, J.H., Park, H., Seok, J.N., Kim, C.G., Lee, H.C., and Choi, S.W. (2012, January 17\u201321). Implementation of Sensor-embedded Main Wing Model of Ultra Light Airplane for Health and Usage Monitoring System (HUMS) Test-bed. Proceedings of the 2012 12th International Conference on Control, Automation and Systems (ICCAS), Jeju, Korea."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"107553","DOI":"10.1016\/j.ast.2022.107553","article-title":"Deep clustering variational network for helicopter regime recognition in HUMS","volume":"124","author":"Wu","year":"2022","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"8364","DOI":"10.1109\/JSEN.2020.2980596","article-title":"Adaptive Channel Weighted CNN With Multisensor Fusion for Condition Monitoring of Helicopter Transmission System","volume":"20","author":"Li","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_9","unstructured":"Marple, S.L., Marino, C., and Strange, S. (2001, January 13\u201316). Large dynamic range time-frequency signal analysis with application to helicopter Doppler radar data. Proceedings of the Sixth International Symposium on Signal Processing and its Applications, Kuala Lumpur, Malaysia."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"He, D., and Bechhoefer, E. (2008, January 1\u20138). Development and validation of bearing diagnostic and prognostic tools using HUMS condition indicators. Proceedings of the 2008 IEEE Aerospace Conference, Big Sky, MT, USA.","DOI":"10.1109\/AERO.2008.4526603"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.ress.2015.01.021","article-title":"Reliability model for helicopter main gearbox lubrication system using influence diagrams","volume":"139","author":"Rashid","year":"2015","journal-title":"Reliab. Eng. Syst. Safe"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"100552","DOI":"10.1016\/j.tsep.2020.100552","article-title":"Convective heat transfer analysis of a laminated flux focusing magnetic gearbox","volume":"18","author":"Modaresahmadi","year":"2020","journal-title":"Therm. Sci. Eng. Prog."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rashid, H., Khalaji, E., Rasheed, J., and Batunlu, C. (2020, January 16\u201318). Fault Prediction of Wind Turbine Gearbox Based on SCADA Data and Machine Learning. Proceedings of the 2020 10th International Conference on Advanced Computer Information Technologies (ACIT), Deggendorf, Germany.","DOI":"10.1109\/ACIT49673.2020.9208884"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1002\/we.1521","article-title":"Monitoring wind turbine gearboxes","volume":"16","author":"Feng","year":"2013","journal-title":"Wind Energy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"185557","DOI":"10.1109\/ACCESS.2020.3029435","article-title":"Research on Fault Diagnosis of Wind Turbine Based on SCADA Data","volume":"8","author":"Liu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"106233","DOI":"10.1016\/j.ijepes.2020.106233","article-title":"Gearbox oil temperature anomaly detection for wind turbine based on sparse Bayesian probability estimation","volume":"123","author":"Zeng","year":"2020","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3462","DOI":"10.1109\/TEC.2021.3075897","article-title":"Wind Turbine Gearbox Anomaly Detection Based on Adaptive Threshold and Twin Support Vector Machines","volume":"36","author":"Dhiman","year":"2021","journal-title":"IEEE Trans. Energy Convers."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1109\/TII.2016.2607179","article-title":"Wind Turbine Gearbox Failure Identification with Deep Neural Networks","volume":"13","author":"Wang","year":"2017","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Guo, R.J., Zhang, G.B., Zhang, Q., and Zhou, L. (2021, January 17\u201320). Early Fault Detection of Wind Turbine Gearbox Based on Adam-Trained LSTM. Proceedings of the 2021 The 6th International Conference on Power and Renewable Energy, Shanghai, China.","DOI":"10.1109\/ICPRE52634.2021.9635550"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yang, S.Y., and Zheng, X.X. (2021, January 17\u201320). Prediction of Gearbox Oil Temperature of Wind Turbine Based on GRNN-LSTM Combined Model. Proceedings of the 2021 The 6th International Conference on Power and Renewable Energy, Shanghai, China.","DOI":"10.1109\/ICPRE52634.2021.9635285"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6354","DOI":"10.1016\/j.egyr.2021.09.080","article-title":"Condition monitoring and performance forecasting of wind turbines based on denoising autoencoder and novel convolutional neural networks","volume":"7","author":"Jia","year":"2021","journal-title":"Energy Rep."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.robot.2019.01.003","article-title":"Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning","volume":"114","author":"You","year":"2019","journal-title":"Robot. Auton. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1248","DOI":"10.1016\/j.eng.2021.04.027","article-title":"Actor-Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking","volume":"7","author":"Dogru","year":"2021","journal-title":"Engineering"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"100977","DOI":"10.1016\/j.aei.2019.100977","article-title":"Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: A deep reinforcement learning approach","volume":"42","author":"Ding","year":"2019","journal-title":"Adv. Eng. Inform."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"109347","DOI":"10.1016\/j.measurement.2021.109347","article-title":"A new hybrid model based on secondary decomposition, reinforcement learning and SRU network for wind turbine gearbox oil temperature forecasting","volume":"178","author":"Liu","year":"2021","journal-title":"Measurement"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2140","DOI":"10.1109\/TSMCB.2004.832154","article-title":"A new Q-learning algorithm based on the Metropolis criterion","volume":"34","author":"Guo","year":"2004","journal-title":"IEEE Trans. Syst. Man Cybern. Part B"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Luo, F., Zhou, Q., Fuentes, J., Ding, W.C., and Gu, C.H. (2022). A Soar-Based Space Exploration Algorithm for Mobile Robots. Entropy, 24.","DOI":"10.3390\/e24030426"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.jprocont.2018.11.004","article-title":"Continuous Control of a Polymerization System with Deep Reinforcement Learning","volume":"75","author":"Ma","year":"2019","journal-title":"J. Process Control"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2022.3216413","article-title":"Partial Transfer Learning of Multidiscriminator Deep Weighted Adversarial Network in Cross-Mechine Fault Diagnosis","volume":"71","author":"Wang","year":"2022","journal-title":"IEEE Trans. Instrum. Measurement"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"108429","DOI":"10.1016\/j.ress.2022.108429","article-title":"Joint decision-making of parallel machine scheduling restricted in job-machine release time and preventive maintenance with remaining useful life constrains","volume":"222","author":"He","year":"2022","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.ijrefrig.2019.07.018","article-title":"A novel deep reinforcement learning based methodology for short-term HVAC system energy consumption prediction","volume":"107","author":"Liu","year":"2019","journal-title":"Int. J. Refrig."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"109675","DOI":"10.1016\/j.enbuild.2019.109675","article-title":"Study on deep reinforcement learning techniques for building energy consumption forecasting","volume":"208","author":"Liu","year":"2020","journal-title":"Energy Build."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"121492","DOI":"10.1016\/j.energy.2021.121492","article-title":"A novel asynchronous deep reinforcement learning model with adaptive early forecasting method and reward incentive mechanism for short-term load forecasting","volume":"236","author":"Zhang","year":"2021","journal-title":"Energy"},{"key":"ref_35","first-page":"1587","article-title":"Addressing Function Approximation Error in Actor-Critic Methods","volume":"80","author":"Fujimoto","year":"2018","journal-title":"Int. Conf. Mach. Learn."},{"key":"ref_36","first-page":"3084","article-title":"DDPG algorithm based on multiple exponential moving average evaluation","volume":"42","author":"Fan","year":"2021","journal-title":"Comput. Eng. Design"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/10\/1394\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:44:24Z","timestamp":1760143464000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/10\/1394"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,30]]},"references-count":36,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["e24101394"],"URL":"https:\/\/doi.org\/10.3390\/e24101394","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2022,9,30]]}}}