{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T17:08:09Z","timestamp":1766164089814,"version":"3.48.0"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"name":"Korea government","award":["No.2022-0-00187"],"award-info":[{"award-number":["No.2022-0-00187"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2026,2,28]]},"abstract":"<jats:p>Time series classification (TSC) presents significant challenges in data analytics and plays an essential role in industries such as manufacturing, healthcare, and finance. Existing research has utilized sequence models, such as long short-term memory (LSTM) and transformers, to achieve high performance; however, these models fail to fully address the inherent challenges of capturing long-term dependencies in time series data. Recently, efforts have been made to overcome these limitations by processing time series data through conversion into images. One such method, the Gramian Angular Field (GAF), converts time series data into images that visually represent the relationship between time points, effectively capturing global trends that traditional sequence models often miss. However, single-modality approaches, which rely on only one representation of time series data, still face limitations in comprehensively capturing the diverse features of a time series. Additionally, existing multi-modality approaches may overlook the detailed features between time points. To address these issues, this article proposes a patch-level hybrid contrastive learning model that combines transformer with Vision Transformer (ViT). The proposed model converts a one-dimensional time series into GAF images and leverages contrastive learning to robustly learn fine-grained features. This enables the model to effectively capture both temporal relationships between time points and spatial relationships between patches, thereby enhancing its generalization capabilities. Experimental results demonstrate that the proposed model outperforms the existing methods on the UCR TSC dataset. This shows that the model can overcome the limitations of single-modality approaches by integrating the complex structural features of time series data through a multi-modality framework, ultimately leading to improved classification performance. This approach provides a new avenue for enhancing TSC and holds promise for applications across various industries.<\/jats:p>","DOI":"10.1145\/3776550","type":"journal-article","created":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T14:43:36Z","timestamp":1763045016000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Patch-Level Contrastive Learning for Improved Time Series Classification with Gramian Angular Field"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-1312-7350","authenticated-orcid":false,"given":"Yohan","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Data Science, Seoul National University of Science and Technology, Nowon-gu, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-7870-9903","authenticated-orcid":false,"given":"Sungyoung","family":"Yoon","sequence":"additional","affiliation":[{"name":"Life Soft Research &amp; User Experiecne Center, LG Uplus, Yongsan-gu, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-0821-174X","authenticated-orcid":false,"given":"Junho","family":"Shin","sequence":"additional","affiliation":[{"name":"Autonomous Manufacturing Research Center, Korea Electronics Technology Institute, Seongnam, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4199-936X","authenticated-orcid":false,"given":"Younghoon","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, Seoul National University of Science and Technology, Nowon-gu, Republic of Korea"}]}],"member":"320","published-online":{"date-parts":[[2025,12,19]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2021.3082985"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2416723"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.5555\/3000850.3000887"},{"key":"e_1_3_1_5_2","first-page":"1597","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning. PMLR, 1597\u20131607."},{"issue":"2","key":"e_1_3_1_6_2","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1109\/JSSC.2020.3021066","article-title":"High-throughput dynamic time warping accelerator for time-series classification with pipelined mixed-signal time-domain computing","volume":"56","author":"Chen Zhengyu","year":"2020","unstructured":"Zhengyu Chen and Jie Gu. 2020. High-throughput dynamic time warping accelerator for time-series classification with pipelined mixed-signal time-domain computing. IEEE Journal of Solid-State Circuits 56, 2 (2020), 624\u2013635.","journal-title":"IEEE Journal of Solid-State Circuits"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2019.1911747"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-020-00701-z"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2014.2379621"},{"key":"e_1_3_1_10_2","unstructured":"Alexey Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929. Retrieved from https:\/\/arxiv.org\/abs\/2010.11929"},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","unstructured":"Emadeldeen Eldele Mohamed Ragab Zhenghua Chen Min Wu Chee Keong Kwoh Xiaoli Li and Cuntai Guan. 2021. Time-series representation learning via temporal and contextual contrasting. arXiv:2106.14112. Retrieved from https:\/\/arxiv.org\/abs\/2106.14112","DOI":"10.24963\/ijcai.2021\/324"},{"key":"e_1_3_1_12_2","article-title":"Unsupervised scalable representation learning for multivariate time series","volume":"32","author":"Franceschi Jean-Yves","year":"2019","unstructured":"Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. 2019. Unsupervised scalable representation learning for multivariate time series. In Advances in Neural Information Processing Systems, Vol. 32.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_13_2","first-page":"617","volume-title":"Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability","volume":"236","author":"Garcia Gabriel Rodriguez","year":"2022","unstructured":"Gabriel Rodriguez Garcia, Gabriel Michau, M\u00e9lanie Ducoffe, Jayant Sen Gupta, and Olga Fink. 2022. Temporal signals to images: Monitoring the condition of industrial assets with deep learning image processing algorithms. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability 236, 4 (2022), 617\u2013627."},{"key":"e_1_3_1_14_2","first-page":"5321","volume-title":"Proceedings of the 2023 35th Chinese Control and Decision Conference (CCDC)","author":"Guo Lijin","year":"2023","unstructured":"Lijin Guo, Longkang Zhang, and Qilan Huang. 2023. Bearing fault diagnosis based on multi-channel GAF-MTF and Res2Net. In Proceedings of the 2023 35th Chinese Control and Decision Conference (CCDC). IEEE, 5321\u20135327."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_16_2","first-page":"1","volume-title":"Proceedings of the 2023 IEEE PES 15th Asia-Pacific Power and Energy Engineering Conference (APPEEC)","author":"Herath Nirasha","year":"2023","unstructured":"Nirasha Herath, Madhawa Herath, Thakshila Thilakanayake, Chitral Angammana, and Migara Liyanage. 2023. An image-based disaggregation study of time series energy data using Gramian angular field. In Proceedings of the 2023 IEEE PES 15th Asia-Pacific Power and Energy Engineering Conference (APPEEC). IEEE, 1\u20136."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_1_18_2","first-page":"1","volume-title":"Proceedings of the 2023 29th International Conference on Mechatronics and Machine Vision in Practice (M2VIP)","author":"Huang Jing","year":"2023","unstructured":"Jing Huang, Yiming Guo, and Fei He. 2023. Multi-channel data fusion and fault diagnosis of forging press based on GAF-ViT. In Proceedings of the 2023 29th International Conference on Mechatronics and Machine Vision in Practice (M2VIP). IEEE, 1\u20136."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.measurement.2011.02.017"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-020-00710-y"},{"key":"e_1_3_1_21_2","first-page":"284","volume-title":"Proceedings of the 2022 14th International Conference on Advanced Computational Intelligence (ICACI)","author":"Jiang Hao","year":"2022","unstructured":"Hao Jiang, Lianguang Liu, and Cheng Lian. 2022. Multi-modal fusion transformer for multivariate time series classification. In Proceedings of the 2022 14th International Conference on Advanced Computational Intelligence (ICACI). IEEE, 284\u2013288."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2779939"},{"issue":"4","key":"e_1_3_1_23_2","doi-asserted-by":"crossref","first-page":"2803","DOI":"10.1007\/s42835-024-01844-5","article-title":"Hybrid transformer for anomaly detection on railway HVAC systems through feature ensemble of spatial\u2013temporal with multi-channel GADF images","volume":"19","author":"Kim Beomjun","year":"2024","unstructured":"Beomjun Kim, Jeong Won Kang, Chul-Su Kim, Oh Kuen Kwon, and Jeonghwan Gwak. 2024. Hybrid transformer for anomaly detection on railway HVAC systems through feature ensemble of spatial\u2013temporal with multi-channel GADF images. Journal of Electrical Engineering & Technology 19, 4 (2024), 2803\u20132815.","journal-title":"Journal of Electrical Engineering & Technology"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i12.29299"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2021.3077049"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2017.09.066"},{"key":"e_1_3_1_27_2","first-page":"8748","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning. PMLR, 8748\u20138763."},{"key":"e_1_3_1_28_2","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1016\/j.isprsjprs.2020.06.006","article-title":"Self-attention for raw optical satellite time series classification","volume":"169","author":"Ru\u00dfwurm Marc","year":"2020","unstructured":"Marc Ru\u00dfwurm and Marco K\u00f6rner. 2020. Self-attention for raw optical satellite time series classification. ISPRS Journal of Photogrammetry and Remote Sensing 169 (2020), 421\u2013435.","journal-title":"ISPRS Journal of Photogrammetry and Remote Sensing"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-014-0377-7"},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1109\/ASPCON49795.2020.9276717","volume-title":"Proceedings of the 2020 IEEE Applied Signal Processing Conference (ASPCON)","author":"Shankar Anand","year":"2020","unstructured":"Anand Shankar, Hnin Kay Khaing, Samarendra Dandapat, and Shovan Barma. 2020. Epileptic seizure classification based on Gramian angular field transformation and deep learning. In Proceedings of the 2020 IEEE Applied Signal Processing Conference (ASPCON). IEEE, 147\u2013151."},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_1_32_2","first-page":"2835","volume-title":"Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining","volume":"2","author":"Tian Tian","year":"2025","unstructured":"Tian Tian, Chunyan Miao, and Hangwei Qian. 2025. FreRA: A frequency-refined augmentation for contrastive learning on time series classification. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Vol. 2, 2835\u20132846."},{"key":"e_1_3_1_33_2","unstructured":"Sana Tonekaboni Danny Eytan and Anna Goldenberg. 2021. Unsupervised representation learning for time series with temporal neighborhood coding. arXiv:2106.00750. Retrieved from https:\/\/arxiv.org\/abs\/2106.00750"},{"key":"e_1_3_1_34_2","article-title":"Attention is all you need","author":"Vaswani A.","year":"2017","unstructured":"A. Vaswani. 2017. Attention is all you need. In Advances in Neural Information Processing Systems.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_35_2","unstructured":"Zhiguang Wang and Tim Oates. 2015. Imaging time-series to improve classification and imputation. arXiv:1506.00327. Retrieved from https:\/\/arxiv.org\/abs\/1506.00327"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","first-page":"1578","DOI":"10.1109\/IJCNN.2017.7966039","volume-title":"Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN)","author":"Wang Zhiguang","year":"2017","unstructured":"Zhiguang Wang, Weizhong Yan, and Tim Oates. 2017. Time series classification from scratch with deep neural networks: A strong baseline. In Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, 1578\u20131585."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3032699"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2021.3071227"},{"issue":"4","key":"e_1_3_1_39_2","first-page":"206","article-title":"Autoencoder with Gramian angular summation field for anomaly detection in multivariate time series data","volume":"2","author":"Yokkampon Umaporn","year":"2022","unstructured":"Umaporn Yokkampon, Abbe Mowshowitz, Sakmongkon Chumkamon, and Eiji Hayashi. 2022. Autoencoder with Gramian angular summation field for anomaly detection in multivariate time series data. Journal of Advances in Artificial Life Robotics 2, 4 (2022), 206\u2013210.","journal-title":"Journal of Advances in Artificial Life Robotics"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i8.20881"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467401"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i12.17325"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3776550","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T17:03:34Z","timestamp":1766163814000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3776550"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,19]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,2,28]]}},"alternative-id":["10.1145\/3776550"],"URL":"https:\/\/doi.org\/10.1145\/3776550","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"type":"print","value":"2157-6904"},{"type":"electronic","value":"2157-6912"}],"subject":[],"published":{"date-parts":[[2025,12,19]]},"assertion":[{"value":"2024-11-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}