{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T19:48:13Z","timestamp":1771703293174,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,10]],"date-time":"2021-03-10T00:00:00Z","timestamp":1615334400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100008982","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CISE-1937460"],"award-info":[{"award-number":["CISE-1937460"]}],"id":[{"id":"10.13039\/501100008982","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Systems experiencing high-rate dynamic events, termed high-rate systems, typically undergo accelerations of amplitudes higher than 100 g-force in less than 10 ms. Examples include adaptive airbag deployment systems, hypersonic vehicles, and active blast mitigation systems. Given their critical functions, accurate and fast modeling tools are necessary for ensuring the target performance. However, the unique characteristics of these systems, which consist of (1) large uncertainties in the external loads, (2) high levels of non-stationarities and heavy disturbances, and (3) unmodeled dynamics generated from changes in system configurations, in combination with the fast-changing environments, limit the applicability of physical modeling tools. In this paper, a deep learning algorithm is used to model high-rate systems and predict their response measurements. It consists of an ensemble of short-sequence long short-term memory (LSTM) cells which are concurrently trained. To empower multi-step ahead predictions, a multi-rate sampler is designed to individually select the input space of each LSTM cell based on local dynamics extracted using the embedding theorem. The proposed algorithm is validated on experimental data obtained from a high-rate system. Results showed that the use of the multi-rate sampler yields better feature extraction from non-stationary time series compared with a more heuristic method, resulting in significant improvement in step ahead prediction accuracy and horizon. The lean and efficient architecture of the algorithm results in an average computing time of 25 \u03bc\u03bcs, which is below the maximum prediction horizon, therefore demonstrating the algorithm\u2019s promise in real-time high-rate applications.<\/jats:p>","DOI":"10.3390\/s21061954","type":"journal-article","created":{"date-parts":[[2021,3,10]],"date-time":"2021-03-10T20:51:42Z","timestamp":1615409502000},"page":"1954","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Multi-Time Resolution Ensemble LSTMs for Enhanced Feature Extraction in High-Rate Time Series"],"prefix":"10.3390","volume":"21","author":[{"given":"Vahid","family":"Barzegar","sequence":"first","affiliation":[{"name":"Department of Civil, Construction, and Environmental Engineering, Iowa State University, 813 Bissell Road, Ames, IA 50011, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0601-9664","authenticated-orcid":false,"given":"Simon","family":"Laflamme","sequence":"additional","affiliation":[{"name":"Department of Civil, Construction, and Environmental Engineering, Iowa State University, 813 Bissell Road, Ames, IA 50011, USA"},{"name":"Department of Electrical and Computer Engineering, Iowa State University, Ames, IA 50011, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chao","family":"Hu","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Iowa State University, Ames, IA 50011, USA"},{"name":"Department of Mechanical Engineering, Iowa State University, Ames, IA 50011, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0852-5130","authenticated-orcid":false,"given":"Jacob","family":"Dodson","sequence":"additional","affiliation":[{"name":"Air Force Research Laboratory, Munitions Directorate, Fuzes Branch, Eglin Air Force Base, FL 32542, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e2159","DOI":"10.1002\/stc.2159","article-title":"Study of input space for state estimation of high-rate dynamics","volume":"25","author":"Hong","year":"2018","journal-title":"Struct. Control Health Monit."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hong, J., Laflamme, S., Dodson, J., and Joyce, B. (2018). Introduction to State Estimation of High-Rate System Dynamics. Sensors, 18.","DOI":"10.3390\/s18010217"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3827463","DOI":"10.1155\/2018\/3827463","article-title":"An experimental test bed for developing high-rate structural health monitoring methods","volume":"2018","author":"Joyce","year":"2018","journal-title":"Shock Vib."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"106551","DOI":"10.1016\/j.ymssp.2019.106551","article-title":"Millisecond model updating for structures experiencing unmodeled high-rate dynamic events","volume":"138","author":"Downey","year":"2020","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_5","unstructured":"Yan, J., Laflamme, S., Hong, J., and Dodson, J. Online Parameter Estimation under Non-Persistent Excitations for High-Rate Dynamic Systems. Mech. Syst. Signal Process., In Review."},{"key":"ref_6","first-page":"20170844","article-title":"Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks","volume":"474","author":"Vlachas","year":"2018","journal-title":"Proc. R. Soc. A Math. Phys. Eng. Sci."},{"key":"ref_7","unstructured":"Lim, B., and Zohren, S. (2004). Time Series Forecasting with Deep Learning: A Survey. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2130001","DOI":"10.1142\/S0129065721300011","article-title":"An Experimental Review on Deep Learning Architectures for Time Series Forecasting","volume":"31","author":"Riquelme","year":"2021","journal-title":"Int. J. Neural Syst."},{"key":"ref_9","unstructured":"Sen, R., Yu, H.F., and Dhillon, I. (2019). Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13059-019-1837-6","article-title":"DeepImpute: An accurate, fast, and scalable deep neural network method to impute single-cell RNA-seq data","volume":"20","author":"Arisdakessian","year":"2019","journal-title":"Genome Biol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Fayyad, J., Jaradat, M.A., Gruyer, D., and Najjaran, H. (2020). Deep Learning Sensor Fusion for Autonomous Vehicle Perception and Localization: A Review. Sensors, 20.","DOI":"10.3390\/s20154220"},{"key":"ref_12","first-page":"1","article-title":"A Survey of Deep Learning Applications to Autonomous Vehicle Control","volume":"22","author":"Kuutti","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rengasamy, D., Jafari, M., Rothwell, B., Chen, X., and Figueredo, G.P. (2020). Deep Learning with Dynamically Weighted Loss Function for Sensor-Based Prognostics and Health Management. Sensors, 20.","DOI":"10.3390\/s20030723"},{"key":"ref_14","first-page":"733","article-title":"Autonomous vision-based damage chronology for spatiotemporal condition assessment of civil infrastructure using unmanned aerial vehicle","volume":"25","author":"Mondal","year":"2020","journal-title":"Smart Struct. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1111\/mice.12580","article-title":"Uncertainty-assisted deep vision structural health monitoring","volume":"36","author":"Sajedi","year":"2020","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Graves, A. (2012). Supervised Sequence Labelling. Studies in Computational Intelligence, Springer.","DOI":"10.1007\/978-3-642-24797-2_2"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2222","DOI":"10.1109\/TNNLS.2016.2582924","article-title":"LSTM: A Search Space Odyssey","volume":"28","author":"Greff","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhu, L., and Laptev, N. (2017, January 18\u201321). Deep and Confident Prediction for Time Series at Uber. Proceedings of the 2017 IEEE International Conference on Data Mining Workshops (ICDMW), New Orleans, LA, USA.","DOI":"10.1109\/ICDMW.2017.19"},{"key":"ref_19","unstructured":"Guen, V.L., and Thome, N. (2019). Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"102674","DOI":"10.1016\/j.trc.2020.102674","article-title":"Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Forecasting Network-wide Traffic State with Missing Values","volume":"118","author":"Cui","year":"2020","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Vochozka, M., Vrbka, J., and Suler, P. (2020). Bankruptcy or Success? The Effective Prediction of a Company\u2019s Financial Development Using LSTM. Sustainability, 12.","DOI":"10.3390\/su12187529"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lai, G., Chang, W.C., Yang, Y., and Liu, H. (2017). Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. arXiv.","DOI":"10.1145\/3209978.3210006"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"113082","DOI":"10.1016\/j.eswa.2019.113082","article-title":"DSTP-RNN: A dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction","volume":"143","author":"Liu","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_24","unstructured":"Gilpin, W. (2002). Deep reconstruction of strange attractors from time series. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1109\/MCOM.2019.1800155","article-title":"Deep Learning with Long Short-Term Memory for Time Series Prediction","volume":"57","author":"Hua","year":"2019","journal-title":"IEEE Commun. Mag."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.1016\/j.jcp.2018.10.024","article-title":"Deep learning algorithm for data-driven simulation of noisy dynamical system","volume":"376","author":"Yeo","year":"2019","journal-title":"J. Comput. Phys."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","article-title":"Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations","volume":"378","author":"Raissi","year":"2019","journal-title":"J. Comput. Phys."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Diligenti, M., Roychowdhury, S., and Gori, M. (2017, January 18\u201321). Integrating Prior Knowledge into Deep Learning. Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico.","DOI":"10.1109\/ICMLA.2017.00-37"},{"key":"ref_29","unstructured":"Karpatne, A., Watkins, W., Read, J., and Kumar, V. (2017). Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1038\/nature14422","article-title":"Robots that can adapt like animals","volume":"521","author":"Cully","year":"2015","journal-title":"Nature"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1016\/j.procir.2018.03.046","article-title":"Optimisation of manufacturing process parameters using deep neural networks as surrogate models","volume":"72","author":"Pfrommer","year":"2018","journal-title":"Procedia CIRP"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"5015","DOI":"10.1007\/s00521-018-3927-x","article-title":"Variable input observer for nonstationary high-rate dynamic systems","volume":"32","author":"Hong","year":"2018","journal-title":"Neural Comput. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Takens, F. (1981). Detecting strange attractors in turbulence. Dynamical Systems and Turbulence, Warwick 1980, Springer.","DOI":"10.1007\/BFb0091924"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Li, F., Shirahama, K., Nisar, M.A., Huang, X., and Grzegorzek, M. (2020). Deep Transfer Learning for Time Series Data Based on Sensor Modality Classification. Sensors, 20.","DOI":"10.3390\/s20154271"},{"key":"ref_35","unstructured":"Holger Kantz, T.S. (2006). Nonlinear Time Series Analysis, Cambridge University Press."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1007\/s003329900072","article-title":"Delay Embeddings for Forced Systems. I. Deterministic Forcing","volume":"9","author":"Stark","year":"1999","journal-title":"J. Nonlinear Sci."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1023\/A:1026753605784","article-title":"On an Embedding Theorem","volume":"88","author":"Caballero","year":"2000","journal-title":"Acta Math. Hung."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1007\/s00332-003-0534-4","article-title":"Delay Embeddings for Forced Systems. II. Stochastic Forcing","volume":"13","author":"Stark","year":"2003","journal-title":"J. Nonlinear Sci."},{"key":"ref_39","unstructured":"Belghazi, M.I., Baratin, A., Rajeshwar, S., Ozair, S., Bengio, Y., Courville, A., and Hjelm, D. (2018, January 10\u201315). Mutual information neural estimation. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1103\/PhysRevA.33.1134","article-title":"Independent coordinates for strange attractors from mutual information","volume":"33","author":"Fraser","year":"1986","journal-title":"Phys. Rev. A"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A Survey on Transfer Learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"M\u00fcller, M. (2007). Dynamic Time Warping. Information Retrieval for Music and Motion, Springer.","DOI":"10.1007\/978-3-540-74048-3"},{"key":"ref_43","first-page":"26","article-title":"Hyperparameter optimization for machine learning models based on Bayesian optimization","volume":"17","author":"Wu","year":"2019","journal-title":"J. Electron. Sci. Technol."},{"key":"ref_44","unstructured":"Perrone, V., Shen, H., Seeger, M., Archambeau, C., and Jenatton, R. (2019). Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Kim, J.Y., and Cho, S.B. (2019, January 10\u201313). Evolutionary Optimization of Hyperparameters in Deep Learning Models. Proceedings of the 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand.","DOI":"10.1109\/CEC.2019.8790354"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1089\/big.2020.0051","article-title":"Coronavirus Optimization Algorithm: A Bioinspired Metaheuristic Based on the COVID-19 Propagation Model","volume":"8","author":"Torres","year":"2020","journal-title":"Big Data"},{"key":"ref_47","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/1954\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:33:47Z","timestamp":1760160827000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/1954"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,10]]},"references-count":47,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21061954"],"URL":"https:\/\/doi.org\/10.3390\/s21061954","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,10]]}}}