{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T06:31:17Z","timestamp":1775025077297,"version":"3.50.1"},"reference-count":55,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T00:00:00Z","timestamp":1722297600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JSAN"],"abstract":"<jats:p>Tool condition monitoring (TCM) is crucial in the machining process to confirm product quality as well as process efficiency and minimize downtime. Traditional methods for TCM, while effective to a degree, often fall short in real-time adaptability and predictive accuracy. This research work aims to advance the state-of-the-art methods in predictive maintenance for TCM and improve tool performance and reliability during the milling process. The present work investigates the application of Deep Learning (DL) and Reinforcement Learning (RL) techniques to monitor tool conditions in milling operations. DL models, including Long Short-Term Memory (LSTM) networks, Feed Forward Neural Networks (FFNN), and RL models, including Q-learning and SARSA, are employed to classify tool conditions from the vibration sensor. The performance of the selected DL and RL algorithms is evaluated through performance metrics like confusion matrix, recall, precision, F1 score, and Receiver Operating Characteristics (ROC) curves. The results revealed that RL based on SARSA outperformed other algorithms. The overall classification accuracies for LSTM, FFNN, Q-learning, and SARSA were 94.85%, 98.16%, 98.50%, and 98.66%, respectively. In regard to predicting tool conditions accurately and thereby enhancing overall process efficiency, SARSA showed the best performance, followed by Q-learning, FFNN, and LSTM. This work contributes to the advancement of TCM systems, highlighting the potential of DL and RL techniques to revolutionize manufacturing processes in the era of Industry 5.0.<\/jats:p>","DOI":"10.3390\/jsan13040042","type":"journal-article","created":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T11:02:27Z","timestamp":1722337347000},"page":"42","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Tool Condition Monitoring in the Milling Process Using Deep Learning and Reinforcement Learning"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4983-8184","authenticated-orcid":false,"given":"Devarajan","family":"Kaliyannan","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Coimbatore 641112, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4866-1428","authenticated-orcid":false,"given":"Mohanraj","family":"Thangamuthu","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Coimbatore 641112, India"}]},{"given":"Pavan","family":"Pradeep","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Coimbatore 641112, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3877-3063","authenticated-orcid":false,"given":"Sakthivel","family":"Gnansekaran","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Vellore Institute of Technology, Chennai 600127, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4144-827X","authenticated-orcid":false,"given":"Jegadeeshwaran","family":"Rakkiyannan","sequence":"additional","affiliation":[{"name":"Centre for Automation, School of Mechanical Engineering, Vellore Institute of Technology, Chennai 600127, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8985-7358","authenticated-orcid":false,"given":"Alokesh","family":"Pramanik","sequence":"additional","affiliation":[{"name":"School of Civil and Mechanical Engineering, Curtin University, Perth 6102, Australia"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"092002","DOI":"10.1088\/1361-6501\/ad519b","article-title":"Review of advances in tool condition monitoring techniques in the milling process","volume":"35","author":"Mohanraj","year":"2024","journal-title":"Meas. Sci. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1032","DOI":"10.1016\/j.jmrt.2019.10.031","article-title":"Tool condition monitoring techniques in milling process\u2014A review","volume":"9","author":"Mohanraj","year":"2020","journal-title":"J. Mater. Res. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/S0166-3615(96)00075-9","article-title":"A review of machine vision sensors for tool condition monitoring","volume":"34","author":"Kurada","year":"1997","journal-title":"Comput. Ind."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"140726","DOI":"10.1109\/ACCESS.2020.3010378","article-title":"Milling tool wear prediction method based on deep learning under variable working conditions","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"108671","DOI":"10.1016\/j.measurement.2020.108671","article-title":"Development of tool condition monitoring system in end milling process using wavelet features and Hoelder\u2019s exponent with machine learning algorithms","volume":"173","author":"Mohanraj","year":"2021","journal-title":"Measurement"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chennai Viswanathan, P., Venkatesh, S.N., Dhanasekaran, S., Mahanta, T.K., Sugumaran, V., Lakshmaiya, N., Paramasivam, P., and Nanjagoundenpalayam Ramasamy, S. (2023). Deep learning for enhanced fault diagnosis of monoblock centrifugal pumps: Spectrogram-based analysis. Machines, 11.","DOI":"10.3390\/machines11090874"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1080\/0951192X.2018.1550681","article-title":"Prediction of cutting tool wear during milling process using artificial intelligence techniques","volume":"32","author":"Shankar","year":"2019","journal-title":"Int. J. Comput. Integr. Manuf."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nair, V.S., Rameshkumar, K., and Saravanamurugan, S. (2024). Chatter Identification in Milling of Titanium Alloy Using Machine Learning Approaches with Non-Linear Features of Cutting Force and Vibration Signatures. Int. J. Progn. Health Manag., 15.","DOI":"10.36001\/ijphm.2024.v15i1.3590"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"95491","DOI":"10.1109\/ACCESS.2020.2995586","article-title":"Tool wear condition monitoring in milling process based on current sensors","volume":"8","author":"Zhou","year":"2020","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"203902","DOI":"10.1016\/j.wear.2021.203902","article-title":"Research on tool wear prediction based on temperature signals and deep learning","volume":"478","author":"He","year":"2021","journal-title":"Wear"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5111","DOI":"10.1007\/s00170-024-13680-y","article-title":"Enhanced tool condition monitoring using wavelet transform-based hybrid deep learning based on sensor signal and vision system","volume":"132","author":"Abdeltawab","year":"2024","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1007\/s00138-004-0137-6","article-title":"Tool wear monitoring using a fast Hough transform of images of machined surfaces","volume":"15","author":"Mannan","year":"2004","journal-title":"Mach. Vis. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4027","DOI":"10.1007\/s00170-023-10811-9","article-title":"Tool remaining useful life prediction using bidirectional recurrent neural networks (BRNN)","volume":"125","author":"Ferrando","year":"2023","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Natarajan, S., Thangamuthu, M., Gnanasekaran, S., and Rakkiyannan, J. (2023). Digital twin-driven tool condition monitoring for the milling process. Sensors, 23.","DOI":"10.3390\/s23125431"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"113937","DOI":"10.1016\/j.measurement.2023.113937","article-title":"Real-time monitoring and measurement of energy characteristics in sustainable machining of titanium alloys","volume":"224","author":"Gupta","year":"2024","journal-title":"Measurement"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Arendra, A., Herianto, H., Akhmad, S., and Lumintu, I. (2021). Dimensions Reduction of Vibration Signal Features Using LDA and PCA for Real Time Tool Wear Detection with Single Layer Perceptron. IOP Conference Series: Materials Science and Engineering, IOP Publishing.","DOI":"10.1088\/1757-899X\/1125\/1\/012052"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1049\/cit2.12180","article-title":"Deep learning: Applications, architectures, models, tools, and frameworks: A comprehensive survey","volume":"8","author":"Gheisari","year":"2023","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.1016\/j.procir.2022.05.110","article-title":"ConvLSTM deep learning signal prediction for forecasting bending moment for tool condition monitoring","volume":"107","author":"Hall","year":"2022","journal-title":"Procedia CIRP"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1049\/cit2.12110","article-title":"A robust deformed convolutional neural network (CNN) for image denoising","volume":"8","author":"Zhang","year":"2023","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"13629","DOI":"10.1109\/JSEN.2023.3269529","article-title":"Measuring Surface Characteristics in Sustainable Machining of Titanium Alloys Using Deep Learning-Based Image Processing","volume":"23","author":"Ross","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1109\/TNNLS.2018.2832648","article-title":"A cost-sensitive deep belief network for imbalanced classification","volume":"30","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2493","DOI":"10.1007\/s00170-019-04788-7","article-title":"An unsupervised online monitoring method for tool wear using a sparse auto-encoder","volume":"106","author":"Dou","year":"2020","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1497","DOI":"10.1007\/s10845-019-01526-4","article-title":"A hybrid information model based on long short-term memory network for tool condition monitoring","volume":"31","author":"Cai","year":"2020","journal-title":"J. Intell. Manuf."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2021.3096283","article-title":"Tool Wear Recognition Based on Deep Kernel Autoencoder with Multichannel Signals Fusion","volume":"70","author":"Ou","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1049\/cit2.12150","article-title":"A complementary integrated Transformer network for hyperspectral image classification","volume":"8","author":"Liao","year":"2023","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1007\/s10845-023-02074-8","article-title":"A novel approach of tool condition monitoring in sustainable machining of Ni alloy with transfer learning models","volume":"35","author":"Ross","year":"2024","journal-title":"J. Intell. Manuf."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1016\/j.jmsy.2021.06.006","article-title":"Intelligent tool wear monitoring based on parallel residual and stacked bidirectional long short-term memory network","volume":"60","author":"Liu","year":"2021","journal-title":"J. Manuf. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3881","DOI":"10.1007\/s00170-023-11017-9","article-title":"Tool wear monitoring based on the combination of machine vision and acoustic emission","volume":"125","author":"Chen","year":"2023","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_29","first-page":"74","article-title":"In-process tool condition monitoring based on convolution neural network","volume":"26","author":"Cao","year":"2020","journal-title":"Comput. Integr. Manuf. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Nguyen, V., Nguyen, V., and Pham, V. (2020). Deep Stacked Auto-Encoder Network Based Tool Wear Monitoring in the Face Milling Process. J. Mech. Eng.\/Stroj. Vestn., 66.","DOI":"10.5545\/sv-jme.2019.6285"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"108554","DOI":"10.1016\/j.measurement.2020.108554","article-title":"Tool wear mechanism and prediction in milling TC18 titanium alloy using deep learning","volume":"173","author":"Ma","year":"2021","journal-title":"Measurement"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Khandey, U., and Arya, V. (2023). Optimization of Multiple Surface Roughness Characteristics of Mild Steel Turned Product Using Weighted Principal Component and Taguchi Method. Materials Today: Proceedings, Elsevier.","DOI":"10.1016\/j.matpr.2023.04.298"},{"key":"ref_33","unstructured":"Tran, Q.K., Huynh, K.T., Grall, A., Langeron, Y., and Mosayebi Omshi, E. (2023). A Review on Reinforcement Learning in Condition-Based Maintenance, IDEALS."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"953","DOI":"10.1007\/s00170-020-05449-w","article-title":"Review of tool condition monitoring in machining and opportunities for deep learning","volume":"109","author":"Serin","year":"2020","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Gnanasekaran, S., Jakkamputi, L.P., Rakkiyannan, J., Thangamuthu, M., and Bhalerao, Y. (2023). A comprehensive approach for detecting brake pad defects using histogram and wavelet features with nested dichotomy family classifiers. Sensors, 23.","DOI":"10.3390\/s23229093"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1073","DOI":"10.1016\/S0890-6955(99)00122-4","article-title":"Sensor signals for tool-wear monitoring in metal cutting operations\u2014A review of methods","volume":"40","author":"Dimla","year":"2000","journal-title":"Int. J. Mach. Tools Manuf."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/0043-1648(83)90256-9","article-title":"An investigation of tool wear and the vibration spectrum in milling","volume":"91","author":"Mehta","year":"1983","journal-title":"Wear"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1007\/s00170-007-0986-z","article-title":"Development of a cutting tool condition monitoring system for high speed turning operation by vibration and strain analysis","volume":"37","author":"Chelladurai","year":"2008","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1007\/s001700200080","article-title":"The correlation of vibration signal features to cutting tool wear in a metal turning operation","volume":"19","author":"Dimla","year":"2002","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1016\/0890-6955(95)00058-5","article-title":"Tool condition monitoring in drilling using vibration signature analysis","volume":"36","author":"Gao","year":"1996","journal-title":"Int. J. Mach. Tools Manuf."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.ndteint.2006.09.006","article-title":"Tool wear evaluation by vibration analysis during end milling of AISI D3 cold work tool steel with 35 HRC hardness","volume":"40","author":"Orhan","year":"2007","journal-title":"NDT E Int."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1016\/S0890-6955(99)00084-X","article-title":"On-line metal cutting tool condition monitoring.: I: Force and vibration analyses","volume":"40","author":"Dimla","year":"2000","journal-title":"Int. J. Mach. Tools Manuf."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"4023","DOI":"10.1007\/s00170-022-08861-6","article-title":"Tool wear monitoring for cavity milling based on vibration singularity analysis and stacked LSTM","volume":"120","author":"Ma","year":"2022","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"612","DOI":"10.17531\/ein.2021.4.3","article-title":"Tool wear condition monitoring in milling process based on data fusion enhanced long short-term memory network under different cutting conditions","volume":"23","author":"Zheng","year":"2021","journal-title":"Eksploat. I Niezawodn."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"810","DOI":"10.1007\/s11227-021-03903-4","article-title":"Tool wear prediction using convolutional bidirectional LSTM networks","volume":"78","author":"Chan","year":"2022","journal-title":"J. Supercomput."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Chen, Q., Xie, Q., Yuan, Q., Huang, H., and Li, Y. (2019). Research on a real-time monitoring method for the wear state of a tool based on a convolutional bidirectional LSTM model. Symmetry, 11.","DOI":"10.3390\/sym11101233"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1917","DOI":"10.1007\/s00170-018-2571-z","article-title":"Predicting tool wear with multi-sensor data using deep belief networks","volume":"99","author":"Chen","year":"2018","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"37","DOI":"10.23940\/ijpe.22.01.p5.3746","article-title":"Cutting tool condition monitoring using a deep learning-based artificial neural network","volume":"18","author":"Patil","year":"2022","journal-title":"Int. J. Perform. Eng."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Ou, J., Li, H., Huang, G., and Zhou, Q. (2020). A Novel Order Analysis and Stacked Sparse Auto-Encoder Feature Learning Method for Milling Tool Wear Condition Monitoring. Sensors, 20.","DOI":"10.3390\/s20102878"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"12885","DOI":"10.1007\/s10462-023-10468-6","article-title":"Reinforcement learning for predictive maintenance: A systematic technical review","volume":"56","author":"Siraskar","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"100977","DOI":"10.1016\/j.aei.2019.100977","article-title":"Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: A deep reinforcement learning approach","volume":"42","author":"Ding","year":"2019","journal-title":"Adv. Eng. Inform."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"103487","DOI":"10.1016\/j.advengsoft.2023.103487","article-title":"Applications of Reinforcement Learning for maintenance of engineering systems: A review","volume":"183","year":"2023","journal-title":"Adv. Eng. Softw."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"105254","DOI":"10.1016\/j.ijmecsci.2019.105254","article-title":"Vibration singularity analysis for milling tool condition monitoring","volume":"166","author":"Zhou","year":"2020","journal-title":"Int. J. Mech. Sci."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"96661","DOI":"10.1109\/ACCESS.2020.2996607","article-title":"The Monitoring of Milling Tool Tipping by Estimating Holder Exponents of Vibration","volume":"8","author":"Zhou","year":"2020","journal-title":"IEEE Access"}],"container-title":["Journal of Sensor and Actuator Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2224-2708\/13\/4\/42\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:26:16Z","timestamp":1760109976000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2224-2708\/13\/4\/42"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,30]]},"references-count":55,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["jsan13040042"],"URL":"https:\/\/doi.org\/10.3390\/jsan13040042","relation":{},"ISSN":["2224-2708"],"issn-type":[{"value":"2224-2708","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,30]]}}}