{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T03:23:44Z","timestamp":1790738624261,"version":"4.1.0"},"reference-count":39,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2019,12,27]],"date-time":"2019-12-27T00:00:00Z","timestamp":1577404800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of National Science and Technology (MOST) of Taiwan","award":["106-2221-E-011-106-MY3"],"award-info":[{"award-number":["106-2221-E-011-106-MY3"]}]},{"name":"Ministry of National Science and Technology (MOST)","award":["107-2218-E-011-002"],"award-info":[{"award-number":["107-2218-E-011-002"]}]},{"name":"Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan","award":["Center for Cyber-Physical System Innovation"],"award-info":[{"award-number":["Center for Cyber-Physical System Innovation"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes a framework to perform the sensor classification by using multivariate time series sensors data as inputs. The framework encodes multivariate time series data into two-dimensional colored images, and concatenate the images into one bigger image for classification through a Convolutional Neural Network (ConvNet). This study applied three transformation methods to encode time series into images: Gramian Angular Summation Field (GASF), Gramian Angular Difference Field (GADF), and Markov Transition Field (MTF). Two open multivariate datasets were used to evaluate the impact of using different transformation methods, the sequences of concatenating images, and the complexity of ConvNet architectures on classification accuracy. The results show that the selection of transformation methods and the sequence of concatenation do not affect the prediction outcome significantly. Surprisingly, the simple structure of ConvNet is sufficient enough for classification as it performed equally well with the complex structure of VGGNet. The results were also compared with other classification methods and found that the proposed framework outperformed other methods in terms of classification accuracy.<\/jats:p>","DOI":"10.3390\/s20010168","type":"journal-article","created":{"date-parts":[[2019,12,27]],"date-time":"2019-12-27T11:42:47Z","timestamp":1577446967000},"page":"168","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":141,"title":["Sensor Classification Using Convolutional Neural Network by Encoding Multivariate Time Series as Two-Dimensional Colored Images"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5585-0094","authenticated-orcid":false,"given":"Chao-Lung","family":"Yang","sequence":"first","affiliation":[{"name":"Department of Industrial Management, National Taiwan University of Science and Technology, Taipei City 10607, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhi-Xuan","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Industrial Management, National Taiwan University of Science and Technology, Taipei City 10607, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chen-Yi","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Industrial Management, National Taiwan University of Science and Technology, Taipei City 10607, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,12,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"614","DOI":"10.1080\/02533839.2017.1372223","article-title":"A cyber-physical scheme for predicting tool wear based on a hybrid dynamic neural network","volume":"40","author":"Yang","year":"2017","journal-title":"J. Chin. Inst. Eng."},{"key":"ref_2","unstructured":"Box, G.E., Jenkins, G.M., Reinsel, G.C., and Ljung, G.M. (2015). Time Series Analysis: Forecasting and Control, John Wiley & Sons. [5th ed.]."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.engappai.2010.09.007","article-title":"A review on time series data mining","volume":"24","author":"Fu","year":"2011","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2305","DOI":"10.1016\/j.eswa.2014.11.007","article-title":"Multivariate time series classification with parametric derivative dynamic time warping","volume":"42","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1080\/01621459.1960.10482064","article-title":"Optimal properties of exponentially weighted forecasts","volume":"55","author":"Muth","year":"1960","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_6","unstructured":"Berndt, D.J., and Clifford, J. (2019, January 4\u20138). Using dynamic time warping to find patterns in time series. Proceedings of the KDD Workshop, Anchorage, AK, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1016\/j.rser.2016.01.011","article-title":"A review of the development of Smart Grid technologies","volume":"59","author":"Tuballa","year":"2016","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_8","unstructured":"Batal, I., Sacchi, L., Bellazzi, R., and Hauskrecht, M. (2018, January 21\u201323). Multivariate Time Series Classification with Temporal Abstractions. Proceedings of the FLAIRS Conference, Melbourne, FL, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1007\/s10618-014-0361-2","article-title":"Time series classification with ensembles of elastic distance measures","volume":"29","author":"Lines","year":"2015","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","article-title":"Deep learning for time series classification: A review","volume":"33","author":"Forestier","year":"2019","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.patrec.2014.01.008","article-title":"A review of unsupervised feature learning and deep learning for time-series modeling","volume":"42","author":"Karlsson","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Liu, Q., Chen, E., Ge, Y., and Zhao, J.L. (2014, January 16\u201318). Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks. Proceedings of the International Conference on Web-Age Information Management (WAIM 2014), Macau, China.","DOI":"10.1007\/978-3-319-08010-9_33"},{"key":"ref_13","unstructured":"Gamboa, J.C. (2017). Deep Learning for Time-Series Analysis. arXiv."},{"key":"ref_14","unstructured":"Adeli, H., and Jiang, X. (2008). Intelligent Infrastructure: Neural Networks, Wavelets, and Chaos Theory for Intelligent Transportation Systems and Smart Structures, CRC Press."},{"key":"ref_15","unstructured":"Wang, Z., and Oates, T. (February, January 27). Encoding Time Series as Images for Visual Inspection and Classification Using Tiled Convolutional Neural Networks. Proceedings of the Workshops at AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_16","unstructured":"Wang, Z., and Oates, T. (2015, January 27\u201330). Imaging Time-Series to Improve Classification and Imputation. Proceedings of the 17th International Conference on Artificial Intelligence, Las Vegas, NV, USA."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1016\/j.neucom.2019.06.032","article-title":"A deep learning framework for time series classification using Relative Position Matrix and Convolutional Neural Network","volume":"359","author":"Chen","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2744","DOI":"10.1109\/TITS.2015.2419614","article-title":"Urban traffic flow prediction system using a multifactor pattern recognition model","volume":"16","author":"Kim","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_19","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_20","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_23","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Lauderdale, FL, USA."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1038\/35016072","article-title":"Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit","volume":"405","author":"Hahnloser","year":"2000","journal-title":"Nature"},{"key":"ref_25","unstructured":"Zhou, Y.-T., and Chellappa, R. (1993, January 25\u201329). Computation of optical flow using a neural network. Proceedings of the IEEE International Conference on Neural Networks, Nagoya, Japan."},{"key":"ref_26","unstructured":"Wang, Z., and Oates, T. (2015). Imaging time-series to improve classification and imputation. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.3844\/jcssp.2018.1499.1511","article-title":"Solving Traffic Routing System using VANet Strategy Combined with a Distributed Swarm Intelligence Optimization","volume":"14","author":"Mouhcine","year":"2018","journal-title":"J. Comput. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wan, J., Liu, J., Shao, Z., Vasilakos, A., Imran, M., and Zhou, K.J.S. (2016). Mobile crowd sensing for traffic prediction in internet of vehicles. Sensors, 16.","DOI":"10.3390\/s16010088"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Nagem, T.A., Qahwaji, R., and Ipson, S. (2017, January 18\u201320). Deep learning teachology for the prediction of solar flares from GOES data. Proceedings of the 2017 Computing Conference, Hangzhou, China.","DOI":"10.1109\/SAI.2017.8252172"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1049\/iet-its.2016.0208","article-title":"LSTM network: A deep learning approach for short-term traffic forecast","volume":"11","author":"Zhao","year":"2017","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_31","unstructured":"Olszewski, R.T. (2001). Generalized Feature Extraction for Structural Pattern Recognition in Time-Series Data, Carnegie-Mellon University Pittsburgh PA School of Computer Science."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1007\/PL00011669","article-title":"Dimensionality Reduction for Fast Similarity Search in Large Time Series Databases","volume":"3","author":"Keogh","year":"2001","journal-title":"Knowl. Inf. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Yang, C., Yang, C., Chen, Z., and Lo, N. (2019, January 14\u201316). Multivariate Time Series Data Transformation for Convolutional Neural Network. Proceedings of the 2019 IEEE\/SICE International Symposium on System Integration (SII), Kobe, Japan.","DOI":"10.1109\/SII.2019.8700425"},{"key":"ref_34","unstructured":"Palm, R.B. (2012). Prediction as a Candidate for Learning Deep Hierarchical Models of Data, Technical University of Denmark."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1007\/s00365-006-0663-2","article-title":"On Early Stopping in Gradient Descent Learning","volume":"26","author":"Yao","year":"2007","journal-title":"Constr. Approx."},{"key":"ref_36","unstructured":"Faouzi, J. (2018). Pyts: A python package for time series transformation and classification. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/s10044-013-0351-9","article-title":"Stacking for multivariate time series classification","volume":"18","author":"Prieto","year":"2015","journal-title":"Pattern Anal. Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"373","DOI":"10.3233\/JIFS-171393","article-title":"Combining raw and normalized data in multivariate time series classification with dynamic time warping","volume":"34","year":"2018","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.neunet.2019.04.014","article-title":"Multivariate LSTM-FCNs for time series classification","volume":"116","author":"Karim","year":"2019","journal-title":"Neural Netw."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/168\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:46:04Z","timestamp":1760190364000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/168"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,27]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20010168"],"URL":"https:\/\/doi.org\/10.3390\/s20010168","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,27]]}}}