{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T05:37:58Z","timestamp":1782538678793,"version":"3.54.5"},"reference-count":46,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,9,3]],"date-time":"2018-09-03T00:00:00Z","timestamp":1535932800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51475324"],"award-info":[{"award-number":["51475324"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Natural Science Foundation of China and Civil Aviation Administration of China jointly funded project","award":["U1533103"],"award-info":[{"award-number":["U1533103"]}]},{"name":"The Basic Innovation Project of China North Industries Group Corporation","award":["2017CX031"],"award-info":[{"award-number":["2017CX031"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Data-driven methods with multi-sensor time series data are the most promising approaches for monitoring machine health. Extracting fault-sensitive features from multi-sensor time series is a daunting task for both traditional data-driven methods and current deep learning models. A novel hybrid end-to-end deep learning framework named Time-distributed ConvLSTM model (TDConvLSTM) is proposed in the paper for machine health monitoring, which works directly on raw multi-sensor time series. In TDConvLSTM, the normalized multi-sensor data is first segmented into a collection of subsequences by a sliding window along the temporal dimension. Time-distributed local feature extractors are simultaneously applied to each subsequence to extract local spatiotemporal features. Then a holistic ConvLSTM layer is designed to extract holistic spatiotemporal features between subsequences. At last, a fully-connected layer and a supervised learning layer are stacked on the top of the model to obtain the target. TDConvLSTM can extract spatiotemporal features on different time scales without any handcrafted feature engineering. The proposed model can achieve better performance in both time series classification tasks and regression prediction tasks than some state-of-the-art models, which has been verified in the gearbox fault diagnosis experiment and the tool wear prediction experiment.<\/jats:p>","DOI":"10.3390\/s18092932","type":"journal-article","created":{"date-parts":[[2018,9,5]],"date-time":"2018-09-05T03:08:55Z","timestamp":1536116935000},"page":"2932","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":138,"title":["A Time-Distributed Spatiotemporal Feature Learning Method for Machine Health Monitoring with Multi-Sensor Time Series"],"prefix":"10.3390","volume":"18","author":[{"given":"Huihui","family":"Qiao","sequence":"first","affiliation":[{"name":"Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300350, China"},{"name":"School of Mechanical Engineering, Tianjin University, Tianjin 300354, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taiyong","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300350, China"},{"name":"School of Mechanical Engineering, Tianjin University, Tianjin 300354, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300350, China"},{"name":"School of Mechanical Engineering, Tianjin University, Tianjin 300354, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shibin","family":"Qiao","sequence":"additional","affiliation":[{"name":"Institute for Special Steels, Central Iron and Steel Research Institute, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin 300350, China"},{"name":"School of Mechanical Engineering, Tianjin University, Tianjin 300354, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"17543","DOI":"10.1109\/ACCESS.2017.2741105","article-title":"Industrial Big Data Analysis in Smart Factory: Current Status and Research Strategies","volume":"5","author":"Xu","year":"2017","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1310","DOI":"10.1109\/TII.2016.2645238","article-title":"Dislocated Time Series Convolutional Neural Architecture: An Intelligent Fault Diagnosis Approach for Electric Machine","volume":"13","author":"Liu","year":"2017","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1434","DOI":"10.1109\/TIE.2013.2261033","article-title":"Fault diagnosis in internal combustion engines using extension neural network","volume":"61","author":"Shatnawi","year":"2014","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1693","DOI":"10.1109\/TIM.2017.2669947","article-title":"Multisensor Feature Fusion for Bearing Fault Diagnosis Using Sparse Autoencoder and Deep Belief Network","volume":"66","author":"Chen","year":"2017","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/j.ymssp.2012.06.021","article-title":"Planetary gearbox fault diagnosis using an adaptive stochastic resonance method","volume":"38","author":"Lei","year":"2013","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Jing, L., Wang, T., and Zhao, M. (2017). An Adaptive Multi-Sensor Data Fusion Method Based on Deep Convolutional Neural Networks for Fault Diagnosis of Planetary Gearbox. Sensors, 17.","DOI":"10.3390\/s17020414"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.1109\/TIE.2017.2733438","article-title":"Machine Health Monitoring Using Local Feature-based Gated Recurrent Unit Networks","volume":"65","author":"Zhao","year":"2018","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1007\/s00170-016-9082-6","article-title":"Tool condition monitoring for form milling of large parts by combining spindle motor current and acoustic emission signals","volume":"89","author":"Uekita","year":"2017","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bengio, Y., and Delalleau, O. (2011, January 5\u20137). On the expressive power of deep architectures. Proceedings of the 14th International Conference on Discovery Science, Espoo, Finland.","DOI":"10.1007\/978-3-642-24477-3_1"},{"key":"ref_11","unstructured":"Lee, H., Pham, P., and Largman, Y. (2009, January 7\u201310). Unsupervised feature learning for audio classification using convolutional deep belief networks. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_12","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Carson, NV, USA."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Le, Q.V., Zou, W.Y., and Yeung, S.Y. (2011, January 20\u201325). Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995496"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Singh, S.P., Kumar, A., and Darbari, H. (2017, January 25\u201326). Building Machine Learning System with Deep Neural Network for Text Processing. Proceedings of the International Conference on Information and Communication Technology for Intelligent Systems, Ahmedabad, India.","DOI":"10.1007\/978-3-319-63645-0_56"},{"key":"ref_15","unstructured":"Zhao, R., Yan, R., and Chen, Z. (arXiv, 2016). Deep Learning and Its Applications to Machine Health Monitoring: A Survey, arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhao, G., Zhang, G., and Ge, Q. (2017, January 9\u201312). Research advances in fault diagnosis and prognostic based on deep learning. Proceedings of the Prognostics and System Health Management Conference, Harbin, China.","DOI":"10.1109\/PHM.2016.7819786"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"7067","DOI":"10.1109\/TIE.2016.2582729","article-title":"Real-time motor fault detection by 1-d convolutional neural networks","volume":"63","author":"Ince","year":"2016","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.jsv.2016.10.043","article-title":"Real-time vibration-based structural damage detection using one dimensional convolutional neural networks","volume":"388","author":"Abdeljaber","year":"2017","journal-title":"J. Sound Vib."},{"key":"ref_19","unstructured":"Babu, G.S., Zhao, P., and Li, X.L. (2016, January 16\u201319). Deep convolutional neural network based regression approach for estimation of remaining useful life. Proceedings of the International Conference on Database Systems for Advanced Applications, Dallas, TX, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.nima.2017.06.020","article-title":"Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets","volume":"867","author":"Wielgosz","year":"2017","journal-title":"Nuclear Instrum. Methods Phys. Res. Sect. A"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2451","DOI":"10.1162\/089976600300015015","article-title":"Learning to forget: Continual prediction with lstm","volume":"12","author":"Gers","year":"2000","journal-title":"Neural Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhao, R., Wang, J., Yan, R., and Mao, K. (2016, January 11\u201313). Machine health monitoring with LSTM networks. Proceedings of the 10th International Conference on Sensing Technology (ICST), Nanjing, China.","DOI":"10.1109\/ICSensT.2016.7796266"},{"key":"ref_23","unstructured":"Shi, X., Chen, Z., and Wang, H. (2015, January 7\u201312). Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Proceedings of the 29th Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Luo, W., Liu, W., and Gao, S. (2017, January 10\u201314). Remembering history with convolutional LSTM for anomaly detection. Proceedings of the IEEE International Conference on Multimedia and Expo, Hong Kong, China.","DOI":"10.1109\/ICME.2017.8019325"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Palaz, D., and Collobert, R. (2015, January 6\u201310). Analysis of cnn-based speech recognition system using raw speech as input. Proceedings of the 16th Annual Conference of the International-Speech-Communication-Association, Dresden, Germany.","DOI":"10.21437\/Interspeech.2015-3"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tra, V., Kim, J., and Khan, S.A. (2017). Bearing Fault Diagnosis under Variable Speed Using Convolutional Neural Networks and the Stochastic Diagonal Levenberg-Marquardt Algorithm. Sensors, 17.","DOI":"10.3390\/s17122834"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1926","DOI":"10.1109\/TIM.2017.2674738","article-title":"Energy-Fluctuated Multiscale Feature Learning with Deep ConvNet for Intelligent Spindle Bearing Fault Diagnosis","volume":"66","author":"Ding","year":"2017","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, W., Peng, G., and Li, C. (2016, January 21\u201323). Rolling Element Bearings Fault Intelligent Diagnosis Based on Convolutional Neural Networks Using Raw Sensing Signal. Smart Innovation Systems and Technologies. Proceedings of the 12th International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), Kaohsiung, Taiwan.","DOI":"10.1007\/978-3-319-50212-0_10"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1109\/TSM.2017.2676245","article-title":"A Convolutional Neural Network for Fault Classification and Diagnosis in Semiconductor Manufacturing Processes","volume":"30","author":"Lee","year":"2017","journal-title":"IEEE Trans. Semicond. Manuf."},{"key":"ref_30","unstructured":"Malhotra, P., Ramakrishnan, A., and Anand, G. (arXiv, 2016). LSTM-based encoder-decoder for multi-sensor anomaly detection, arXiv."},{"key":"ref_31","unstructured":"Malhotra, P., Vishnu, T.V., and Ramakrishnan, A. (2016). Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1109\/TNNLS.2016.2551940","article-title":"Railway Track Circuit Fault Diagnosis Using Recurrent Neural Networks","volume":"28","author":"Bruin","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.specom.2015.12.003","article-title":"Maxout neurons for deep convolutional and LSTM neural networks in speech recognition","volume":"77","author":"Cai","year":"2016","journal-title":"Speech Commun."},{"key":"ref_34","unstructured":"Kahou, S.E., Michalski, V., and Konda, K. (2015, January 9\u201313). Recurrent Neural Networks for Emotion Recognition in Video. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, New York, NY, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.neucom.2016.12.088","article-title":"An Analysis of Convolutional Long-Short Term Memory Recurrent Neural Networks for Gesture Recognition","volume":"268","author":"Tsironi","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Rad, N.M., Kia, S.M., Zarbo, C., Jurman, G., Venuti, P., and Furlanello, C. (2017, January 18\u201321). Stereotypical Motor Movement Detection in Dynamic Feature Space. Proceedings of the IEEE International Conference on Data Mining Workshops, New Orleans, LA, USA.","DOI":"10.1109\/ICDMW.2016.0076"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.sigpro.2017.10.011","article-title":"Deep learning for automatic stereotypical motor movement detection using wearable sensors in autism spectrum disorders","volume":"144","author":"Rad","year":"2018","journal-title":"Signal Process."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zhao, R., Yan, R., and Wang, J. (2017). Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks. Sensors, 17.","DOI":"10.3390\/s17020273"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Xiong, F., Shi, X., and Yeung, D.Y. (2017, January 22\u201329). Spatiotemporal Modeling for Crowd Counting in Videos. Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.551"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1109\/TMM.2017.2759504","article-title":"Learning Deep Spatio-Temporal Dependence for Semantic Video Segmentation","volume":"20","author":"Qiu","year":"2017","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_41","unstructured":"(2018, May 26). Understanding LSTM Networks. Available online: http:\/\/colah. github.io\/posts\/2015-08-Understanding-LSTMs\/."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Chan, W., and Jaitly, N. (2017, January 5\u20139). Very Deep Convolutional Networks for End-to-End Speech Recognition. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7953077"},{"key":"ref_43","unstructured":"Ioffe, S., and Szegedy, C. (arXiv, 2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv."},{"key":"ref_44","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_45","unstructured":"Li, X., Lim, B., Zhou, J., Huang, S., Phua, S., Shaw, K., and Er, M. (2009, January 27\u201330). Fuzzy neural network modelling for tool wear estimation in dry milling operation. Proceedings of the Annual Conference of the Prognostics and Health Management Society, San Diego, CA, USA."},{"key":"ref_46","unstructured":"Dozat, T. (2016, January 2\u20134). Incorporating nesterov momentum into adam. Proceedings of the International Conference on Learning Representations, Caribe Hilton, San Juan, Puerto Rico."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2932\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:18:39Z","timestamp":1760195919000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2932"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,3]]},"references-count":46,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["s18092932"],"URL":"https:\/\/doi.org\/10.3390\/s18092932","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,3]]}}}