{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T15:03:22Z","timestamp":1782313402520,"version":"3.54.5"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"5s","license":[{"start":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T00:00:00Z","timestamp":1632268800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["CCF-1942836, CNS-1952096"],"award-info":[{"award-number":["CCF-1942836, CNS-1952096"]}]},{"name":"NRT","award":["1829004"],"award-info":[{"award-number":["1829004"]}]},{"name":"Austrian FFG-funded","award":["IoT4CPS"],"award-info":[{"award-number":["IoT4CPS"]}]},{"DOI":"10.13039\/501100001821","name":"WWTF","doi-asserted-by":"crossref","award":["ICT19-018"],"award-info":[{"award-number":["ICT19-018"]}],"id":[{"id":"10.13039\/501100001821","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>\n            Predictive monitoring\u2014making predictions about future states and monitoring if the predicted states satisfy requirements\u2014offers a promising paradigm in supporting the decision making of Cyber-Physical Systems (CPS). Existing works of predictive monitoring mostly focus on monitoring individual predictions rather than sequential predictions. We develop a novel approach for monitoring sequential predictions generated from Bayesian Recurrent Neural Networks (RNNs) that can capture the inherent uncertainty in CPS, drawing on insights from our study of real-world CPS datasets. We propose a new logic named\n            <jats:italic>Signal Temporal Logic with Uncertainty<\/jats:italic>\n            (STL-U) to monitor a flowpipe containing an infinite set of uncertain sequences predicted by Bayesian RNNs. We define STL-U strong and weak satisfaction semantics based on whether all or some sequences contained in a flowpipe satisfy the requirement. We also develop methods to compute the range of confidence levels under which a flowpipe is guaranteed to strongly (weakly) satisfy an STL-U formula. Furthermore, we develop novel criteria that leverage STL-U monitoring results to calibrate the uncertainty estimation in Bayesian RNNs. Finally, we evaluate the proposed approach via experiments with real-world CPS datasets and a simulated smart city case study, which show very encouraging results of STL-U based predictive monitoring approach outperforming baselines.\n          <\/jats:p>","DOI":"10.1145\/3477032","type":"journal-article","created":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T20:48:40Z","timestamp":1632343720000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Predictive Monitoring with Logic-Calibrated Uncertainty for Cyber-Physical Systems"],"prefix":"10.1145","volume":"20","author":[{"given":"Meiyi","family":"Ma","sequence":"first","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Stankovic","sequence":"additional","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ezio","family":"Bartocci","sequence":"additional","affiliation":[{"name":"TU Wien, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lu","family":"Feng","sequence":"additional","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,9,22]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proc. of RV 2019(LNCS, Vol.\u00a0 11757)","author":"Babaee Reza","year":"2079","unstructured":"Reza Babaee , Vijay Ganesh , and Sean Sedwards . 2019. Accelerated learning of predictive runtime monitors for rare failure . In Proc. of RV 2019(LNCS, Vol.\u00a0 11757) . Springer , 111\u2013128. https:\/\/doi.org\/10.1007\/978-3-030-3 2079 -9 10.1007\/978-3-030-32079-9 Reza Babaee, Vijay Ganesh, and Sean Sedwards. 2019. Accelerated learning of predictive runtime monitors for rare failure. In Proc. of RV 2019(LNCS, Vol.\u00a0 11757). Springer, 111\u2013128. https:\/\/doi.org\/10.1007\/978-3-030-32079-9"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-03769-7_11"},{"key":"#cr-split#-e_1_2_1_3_1.1","doi-asserted-by":"crossref","unstructured":"E. Bartocci J. Deshmukh A. Donz\u00e9 G. Fainekos O. Maler D. Nickovic and S. Sankaranarayanan. 2018. Specification-based Monitoring of Cyber-Physical Systems: A Survey on Theory Tools and Applications. In Lectures on Runtime Verification. LNCS Vol.\u00a0 10457. Springer 135-175. https:\/\/doi.org\/10.1007\/978-3-319-75632-5 10.1007\/978-3-319-75632-5","DOI":"10.1007\/978-3-319-75632-5_5"},{"key":"#cr-split#-e_1_2_1_3_1.2","doi-asserted-by":"crossref","unstructured":"E. Bartocci J. Deshmukh A. Donz\u00e9 G. Fainekos O. Maler D. Nickovic and S. Sankaranarayanan. 2018. Specification-based Monitoring of Cyber-Physical Systems: A Survey on Theory Tools and Applications. In Lectures on Runtime Verification. LNCS Vol.\u00a0 10457. Springer 135-175. https:\/\/doi.org\/10.1007\/978-3-319-75632-5","DOI":"10.1007\/978-3-319-75632-5_5"},{"key":"e_1_2_1_4_1","volume-title":"Proc. of RV 2012(LNCS, Vol.\u00a0 7687)","author":"Bartocci Ezio","year":"2012","unstructured":"Ezio Bartocci , Radu Grosu , Atul Karmarkar , Scott\u00a0 A. Smolka , Scott\u00a0 D. Stoller , Erez Zadok , and Justin Seyster . 2012 . Adaptive Runtime Verification . In Proc. of RV 2012(LNCS, Vol.\u00a0 7687) . Springer, 168\u2013182. https:\/\/doi.org\/10.1007\/978-3-642-35632-2_18 10.1007\/978-3-642-35632-2_18 Ezio Bartocci, Radu Grosu, Atul Karmarkar, Scott\u00a0A. Smolka, Scott\u00a0D. Stoller, Erez Zadok, and Justin Seyster. 2012. Adaptive Runtime Verification. In Proc. of RV 2012(LNCS, Vol.\u00a0 7687). Springer, 168\u2013182. https:\/\/doi.org\/10.1007\/978-3-642-35632-2_18"},{"key":"e_1_2_1_5_1","volume-title":"Proc. of Inter. Conference on Advances in System Simulation. ThinkMind. https:\/\/elib.dlr.de\/71460\/.","author":"Behrisch Michael","year":"2011","unstructured":"Michael Behrisch , Laura Bieker , Jakob Erdmann , and Daniel Krajzewicz . 2011 . SUMO\u2013simulation of urban mobility: an overview . In Proc. of Inter. Conference on Advances in System Simulation. ThinkMind. https:\/\/elib.dlr.de\/71460\/. Michael Behrisch, Laura Bieker, Jakob Erdmann, and Daniel Krajzewicz. 2011. SUMO\u2013simulation of urban mobility: an overview. In Proc. of Inter. Conference on Advances in System Simulation. ThinkMind. https:\/\/elib.dlr.de\/71460\/."},{"key":"e_1_2_1_6_1","volume-title":"Proc. of RV 2019(LNCS, Vol.\u00a0 11757)","author":"Bortolussi Luca","year":"2079","unstructured":"Luca Bortolussi , Francesca Cairoli , Nicola Paoletti , Scott\u00a0 A. Smolka , and Scott\u00a0 D. Stoller . 2019. Neural predictive monitoring . In Proc. of RV 2019(LNCS, Vol.\u00a0 11757) . Springer , 129\u2013147. https:\/\/doi.org\/10.1007\/978-3-030-3 2079 -9_8 10.1007\/978-3-030-32079-9_8 Luca Bortolussi, Francesca Cairoli, Nicola Paoletti, Scott\u00a0A. Smolka, and Scott\u00a0D. Stoller. 2019. Neural predictive monitoring. In Proc. of RV 2019(LNCS, Vol.\u00a0 11757). Springer, 129\u2013147. https:\/\/doi.org\/10.1007\/978-3-030-32079-9_8"},{"key":"e_1_2_1_7_1","volume-title":"Proc. of the 2nd Workshop on Artificial Intelligence and Formal Verification, Logic, Automata, and Synthesis(CEUR Workshop Proceedings, Vol.\u00a0 2785)","author":"Bortolussi Luca","year":"2020","unstructured":"Luca Bortolussi , Francesca Cairoli , Nicola Paoletti , Scott\u00a0 A. Smolka , and Scott\u00a0 D. Stoller . 2020 . Bayesian Neural Predictive Monitoring . In Proc. of the 2nd Workshop on Artificial Intelligence and Formal Verification, Logic, Automata, and Synthesis(CEUR Workshop Proceedings, Vol.\u00a0 2785) . CEUR-WS.org, 95\u2013100. http:\/\/ceur-ws.org\/Vol-2785\/paper16.pdf. Luca Bortolussi, Francesca Cairoli, Nicola Paoletti, Scott\u00a0A. Smolka, and Scott\u00a0D. Stoller. 2020. Bayesian Neural Predictive Monitoring. In Proc. of the 2nd Workshop on Artificial Intelligence and Formal Verification, Logic, Automata, and Synthesis(CEUR Workshop Proceedings, Vol.\u00a0 2785). CEUR-WS.org, 95\u2013100. http:\/\/ceur-ws.org\/Vol-2785\/paper16.pdf."},{"key":"e_1_2_1_8_1","volume-title":"Algorithms for minimization without derivatives","author":"Brent P.","unstructured":"Richard\u00a0 P. Brent . 2013. Algorithms for minimization without derivatives . Courier Corporation . Richard\u00a0P. Brent. 2013. Algorithms for minimization without derivatives. Courier Corporation."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-19249-9_14"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-45069-6_3"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/3045390.3045502"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/3294996.3295116"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/2986459.2986721"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2728606.2728633"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380097"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/168304.168306"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10817-017-9413-9"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40787-1_9"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23180"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295309"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/3304889.3304985"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3358231"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3127041.3127045"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/3304222.3304244"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2019.2915525"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPS48487.2020.00013"},{"key":"e_1_2_1_28_1","volume-title":"NeurIPS","author":"Ma Meiyi","year":"2020","unstructured":"Meiyi Ma , Ji Gao , Lu Feng , and John\u00a0 A. Stankovic . 2020. STLnet: Signal Temporal Logic Enforced Multivariate Recurrent Neural Networks . In NeurIPS 2020 . https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/a7da6ba0505a41b98bd85907244c4c30-Abstract.html. Meiyi Ma, Ji Gao, Lu Feng, and John\u00a0A. Stankovic. 2020. STLnet: Signal Temporal Logic Enforced Multivariate Recurrent Neural Networks. In NeurIPS 2020. https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/a7da6ba0505a41b98bd85907244c4c30-Abstract.html."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPS.2018.00014"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.3.448"},{"key":"e_1_2_1_31_1","volume-title":"Proc. of FORMATS 2004(LNCS, Vol.\u00a0 3253)","author":"Maler O.","unstructured":"O. Maler and D. Ni\u010dkovi\u0107 . 2004. Monitoring Temporal Properties of Continuous Signals . In Proc. of FORMATS 2004(LNCS, Vol.\u00a0 3253) . Springer, 152\u2013166. https:\/\/doi.org\/10.1007\/b100824 10.1007\/b100824 O. Maler and D. Ni\u010dkovi\u0107. 2004. Monitoring Temporal Properties of Continuous Signals. In Proc. of FORMATS 2004(LNCS, Vol.\u00a0 3253). Springer, 152\u2013166. https:\/\/doi.org\/10.1007\/b100824"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1177\/1932296817736074"},{"key":"e_1_2_1_33_1","unstructured":"NYC.gov. 2020. New York City Open Data. https:\/\/nycopendata.socrata.com\/.  NYC.gov. 2020. New York City Open Data. https:\/\/nycopendata.socrata.com\/."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445563"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-57628-8"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46520-3_26"},{"key":"e_1_2_1_37_1","volume-title":"Applied statistics: a handbook of techniques","author":"Sachs Lothar","unstructured":"Lothar Sachs . 2012. Applied statistics: a handbook of techniques . Springer Science & Business Media . Lothar Sachs. 2012. Applied statistics: a handbook of techniques. Springer Science & Business Media."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2016.XII.017"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29860-8_15"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450267.3450531"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33017322"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.2972936"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2017.19"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477032","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3477032","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:46Z","timestamp":1750188646000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477032"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,22]]},"references-count":44,"journal-issue":{"issue":"5s","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3477032"],"URL":"https:\/\/doi.org\/10.1145\/3477032","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,22]]},"assertion":[{"value":"2021-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}