{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T21:44:43Z","timestamp":1783806283939,"version":"3.55.0"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,10,26]],"date-time":"2024-10-26T00:00:00Z","timestamp":1729900800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Army Research Office and was accomplished","award":["W911NF-20-1-0080"],"award-info":[{"award-number":["W911NF-20-1-0080"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Cyber-Phys. Syst."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>Uncertainty in the predictions of learning-enabled components hinders their deployment in safety-critical cyber-physical systems (CPS). A shift from the training distribution of a learning-enabled component (LEC) is one source of uncertainty in the LEC\u2019s predictions. Detection of this shift or out-of-distribution (OOD) detection on individual datapoints has therefore gained attention recently. But in many applications, inputs to CPS form a temporal sequence. Existing techniques for OOD detection in time-series data for CPS either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data for CPS. Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher\u2019s method leads to the proposed detector CODiT with bounded false alarms.<\/jats:p>\n          <jats:p>CODiT performs OOD detection on fixed-length windows of consecutive time-series datapoints by using Fisher value of the input window. We further propose performing OOD detection on real-time time-series traces of variable lengths with bounded false alarms. This can be done by using CODiT to compute Fisher values of the sliding windows in the input trace and combining these values by a merging function. Merging functions such as Harmonic Mean, Arithmetic Mean, Geometric Mean, Bonferroni Method, and so on, can be used to combine Fisher values of the sliding windows in the input trace, and the combined value can be used for OOD detection on the trace with bounded false alarm rate guarantees.<\/jats:p>\n          <jats:p>\n            We illustrate the efficacy of CODiT by achieving state-of-the-art results in two case studies for OOD detection on fixed-length windows. The first one is on an autonomous driving system with perception (or vision) LEC. The second case study is on a medical CPS for walking pattern or GAIT analysis where physiological (non-vision) data is collected with force-sensitive resistors attached to the subject\u2019s body. For OOD detection on variable length traces, we consider the same case studies on the autonomous driving system and medical CPS for GAIT analysis. We report our results with four merging functions on the Fisher values computed by CODiT on the sliding windows of the input trace. We also compare the false alarm rate guarantees by these four merging functions in the autonomous driving system case study. Code, data, and trained models are available at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"url\" xlink:href=\"https:\/\/github.com\/kaustubhsridhar\/time-series-OOD\">https:\/\/github.com\/kaustubhsridhar\/time-series-OOD<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3648005","type":"journal-article","created":{"date-parts":[[2024,2,13]],"date-time":"2024-02-13T13:51:36Z","timestamp":1707832296000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Out-of-distribution Detection in Dependent Data for Cyber-physical Systems with Conformal Guarantees"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-7662-3174","authenticated-orcid":false,"given":"Ramneet","family":"Kaur","sequence":"first","affiliation":[{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3233-1720","authenticated-orcid":false,"given":"Yahan","family":"Yang","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5282-0658","authenticated-orcid":false,"given":"Oleg","family":"Sokolsky","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2672-1132","authenticated-orcid":false,"given":"Insup","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Computer and Information, University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,10,26]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.5555\/2671155"},{"key":"e_1_3_3_3_2","volume-title":"Detection of Abrupt Changes: Theory and Application","year":"1993","unstructured":"Michele Basseville and Igor V. Nikiforov. 1993. Detection of Abrupt Changes: Theory and Application. Vol. 104. Prentice Hall, Englewood Cliffs, NJ."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF03178892"},{"key":"e_1_3_3_5_2","unstructured":"Mariusz Bojarski Davide Del Testa Daniel Dworakowski Bernhard Firner Beat Flepp Prasoon Goyal Lawrence D. Jackel Mathew Monfort Urs Muller Jiakai Zhang Xin Zhang and Jake Zhao. 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPS48487.2020.00024"},{"key":"e_1_3_3_7_2","article-title":"Learning augmentation distributions using transformed risk minimization","author":"Chatzipantazis Evangelos","year":"2021","unstructured":"Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Kostas Daniilidis, and Edgar Dobriban. 2021. Learning augmentation distributions using transformed risk minimization. arXiv preprint arXiv:2111.08190 (2021).","journal-title":"arXiv preprint arXiv:2111.08190"},{"issue":"245","key":"e_1_3_3_8_2","first-page":"1","article-title":"A group-theoretic framework for data augmentation","volume":"21","author":"Chen Shuxiao","year":"2020","unstructured":"Shuxiao Chen, Edgar Dobriban, and Jane H. Lee. 2020. A group-theoretic framework for data augmentation. J. Mach. Learn. Res. 21, 245 (2020), 1\u201371.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0107-6"},{"key":"e_1_3_3_10_2","first-page":"1","volume-title":"Conference on Robot learning","author":"Dosovitskiy Alexey","year":"2017","unstructured":"Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An open urban driving simulator. In Conference on Robot learning. PMLR, 1\u201316."},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477026"},{"key":"e_1_3_3_12_2","unstructured":"R. A. Fisher. 1932. Statistical Methods for Research Workers. (4th ed.) Oliver & Boyd."},{"key":"e_1_3_3_13_2","article-title":"RobustTAD: Robust time series anomaly detection via decomposition and convolutional neural networks","author":"Gao Jingkun","year":"2020","unstructured":"Jingkun Gao, Xiaomin Song, Qingsong Wen, Pichao Wang, Liang Sun, and Huan Xu. 2020. RobustTAD: Robust time series anomaly detection via decomposition and convolutional neural networks. arXiv preprint arXiv:2002.09545 (2020).","journal-title":"arXiv preprint arXiv:2002.09545"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.129"},{"key":"e_1_3_3_15_2","volume-title":"International Conference on Learning Representations","author":"Haroush Matan","year":"2021","unstructured":"Matan Haroush, Tzviel Frostig, Ruth Heller, and Daniel Soudry. 2021. A statistical framework for efficient out of distribution detection in deep neural networks. In International Conference on Learning Representations."},{"key":"e_1_3_3_16_2","doi-asserted-by":"crossref","unstructured":"Jeffrey M. Hausdorff Apinya Lertratanakul Merit E. Cudkowicz Amie L. Peterson David Kaliton and Ary L. Goldberger. 2000. Dynamic markers of altered gait rhythm in amyotrophic lateral sclerosis. J. Appl. Physiol. 88 (2000) 2045\u20132053.","DOI":"10.1152\/jappl.2000.88.6.2045"},{"key":"e_1_3_3_17_2","volume-title":"International Conference on Learning Representations","author":"Hendrycks Dan","year":"2017","unstructured":"Dan Hendrycks and Kevin Gimpel. 2017. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations."},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00982"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576842.3582374"},{"key":"e_1_3_3_20_2","first-page":"148","volume-title":"IEEE\/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE\u201923)","year":"2023","unstructured":"Xiayan Ji, Xian Li, Ahhyun Yuh, Amanda Watson, Claire Kendell, James Weimer, Hajime Nagahara, Teruo Higashino, Teruhiro Mizumoto, Viktor Erd\u00e9lyi, George Demiris, Oleg Sokolsky, and Insup Lee. 2023. Short: Integrated sensing platform for detecting social isolation and loneliness in the elderly community. In IEEE\/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE\u201923). IEEE, 148\u2013152."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3450267.3450535"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-55583-2_6"},{"key":"e_1_3_3_23_2","doi-asserted-by":"crossref","unstructured":"Ramneet Kaur Susmit Jha Anirban Roy Sangdon Park Edgar Dobriban Oleg Sokolsky and Insup Lee. 2022. iDECODe: In-distribution Equivariance for Conformal Out-of-distribution Detection Association for the Advancement of Artificial Intelligence. arxiv:2201.02331 [cs.LG]","DOI":"10.1609\/aaai.v36i7.20670"},{"key":"e_1_3_3_24_2","article-title":"Detecting oods as datapoints with high uncertainty","author":"Kaur Ramneet","year":"2021","unstructured":"Ramneet Kaur, Susmit Jha, Anirban Roy, Sangdon Park, Oleg Sokolsky, and Insup Lee. 2021. Detecting oods as datapoints with high uncertainty. arXiv preprint arXiv:2108.06380 (2021).","journal-title":"arXiv preprint arXiv:2108.06380"},{"key":"e_1_3_3_25_2","article-title":"Are all outliers alike? On understanding the diversity of outliers for detecting OODs","author":"Kaur Ramneet","year":"2021","unstructured":"Ramneet Kaur, Susmit Jha, Anirban Roy, Oleg Sokolsky, and Insup Lee. 2021. Are all outliers alike? On understanding the diversity of outliers for detecting OODs. arXiv preprint arXiv:2103.12628 (2021).","journal-title":"arXiv preprint arXiv:2103.12628"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICAA58325.2023.00011"},{"key":"e_1_3_3_27_2","article-title":"Using semantic information for defining and detecting OOD inputs","author":"Kaur Ramneet","year":"2023","unstructured":"Ramneet Kaur, Xiayan Ji, Souradeep Dutta, Michele Caprio, Yahan Yang, Elena Bernardis, Oleg Sokolsky, and Insup Lee. 2023. Using semantic information for defining and detecting OOD inputs. arXiv preprint arXiv:2302.11019 (2023).","journal-title":"arXiv preprint arXiv:2302.11019"},{"key":"e_1_3_3_28_2","article-title":"Detection of adversarial physical attacks in time-series image data","author":"Kaur Ramneet","year":"2023","unstructured":"Ramneet Kaur, Yiannis Kantaros, Wenwen Si, James Weimer, and Insup Lee. 2023. Detection of adversarial physical attacks in time-series image data. arXiv preprint arXiv:2304.13919 (2023).","journal-title":"arXiv preprint arXiv:2304.13919"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576841.3585931"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10472-013-9381-7"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539408"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2017.23"},{"key":"e_1_3_3_34_2","article-title":"DriftNet: Aggressive driving behavior classification using 3D EfficientNet architecture","author":"Noor Alam","year":"2020","unstructured":"Alam Noor, Bilel Benjdira, Adel Ammar, and Anis Koubaa. 2020. DriftNet: Aggressive driving behavior classification using 3D EfficientNet architecture. arXiv preprint arXiv:2004.11970 (2020).","journal-title":"arXiv preprint arXiv:2004.11970"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00822"},{"key":"e_1_3_3_36_2","article-title":"Efficient out-of-distribution detection using latent space of  \\(\\beta\\) -VAE for cyber-physical systems","author":"Ramakrishna Shreyas","year":"2021","unstructured":"Shreyas Ramakrishna, Zahra Rahiminasab, Gabor Karsai, Arvind Easwaran, and Abhishek Dubey. 2021. Efficient out-of-distribution detection using latent space of \\(\\beta\\) -VAE for cyber-physical systems. arXiv preprint:2108.11800 (2021).","journal-title":"arXiv preprint:2108.11800"},{"key":"e_1_3_3_37_2","article-title":"Automold","author":"Saxena Ujjwal","year":"2018","unstructured":"Ujjwal Saxena. 2018. Automold. Retrieved from https:\/\/github.com\/UjjwalSaxena\/Automold--Road-Augmentation-Library","journal-title":"R"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6247909"},{"key":"e_1_3_3_39_2","article-title":"Towards alternative techniques for improving adversarial robustness: Analysis of adversarial training at a spectrum of perturbations","author":"Sridhar Kaustubh","year":"2022","unstructured":"Kaustubh Sridhar, Souradeep Dutta, Ramneet Kaur, James Weimer, Oleg Sokolsky, and Insup Lee. 2022. Towards alternative techniques for improving adversarial robustness: Analysis of adversarial training at a spectrum of perturbations. arXiv preprint arXiv:2206.06496 (2022).","journal-title":"arXiv preprint arXiv:2206.06496"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2012.02.016"},{"key":"e_1_3_3_41_2","article-title":"CSI: Novelty detection via contrastive learning on distributionally shifted instances","volume":"33","author":"Tack Jihoon","year":"2020","unstructured":"Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. 2020. CSI: Novelty detection via contrastive learning on distributionally shifted instances. Adv. Neural Inf. Process. Syst. 33 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2566468.2566483"},{"key":"e_1_3_3_43_2","first-page":"39","volume-title":"Conformal and Probabilistic Prediction and Applications","author":"Toccaceli Paolo","year":"2017","unstructured":"Paolo Toccaceli and Alexander Gammerman. 2017. Combination of conformal predictors for classification. In Conformal and Probabilistic Prediction and Applications. PMLR, 39\u201361."},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00675"},{"key":"e_1_3_3_45_2","first-page":"768","volume-title":"20th International Conference on Machine Learning (ICML\u201903)","author":"Vovk Vladimir","year":"2003","unstructured":"Vladimir Vovk, Ilia Nouretdinov, and Alexander Gammerman. 2003. Testing exchangeability on-line. In 20th International Conference on Machine Learning (ICML\u201903). 768\u2013775."},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/asaa027"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPS54341.2022.00027"}],"container-title":["ACM Transactions on Cyber-Physical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648005","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3648005","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:56:52Z","timestamp":1750291012000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648005"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,26]]},"references-count":46,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3648005"],"URL":"https:\/\/doi.org\/10.1145\/3648005","relation":{},"ISSN":["2378-962X","2378-9638"],"issn-type":[{"value":"2378-962X","type":"print"},{"value":"2378-9638","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,26]]},"assertion":[{"value":"2023-07-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}