{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:49:44Z","timestamp":1750308584708,"version":"3.41.0"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2017,12,6]],"date-time":"2017-12-06T00:00:00Z","timestamp":1512518400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSF project ExCAPE: Expeditions in Computer Augmented Program Engineering","award":["CCF-1138996"],"award-info":[{"award-number":["CCF-1138996"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2018,1,31]]},"abstract":"<jats:p>\n            Evaluation of industrial embedded control system designs is a time-consuming and imperfect process. While an ideal process would apply a formal verification technique such as model checking or theorem proving, these techniques do not scale to industrial design problems, and it is often difficult to use these techniques to verify performance aspects of control system designs, such as stability or convergence. For industrial designs, engineers rely on testing processes to identify critical or unexpected behaviors. We propose a novel framework called\n            <jats:italic>Underminer<\/jats:italic>\n            to improve the testing process; this is an automated technique to identify nonconverging behaviors in embedded control system designs. Underminer treats the system as a black box and lets the designer indicate the model parameters, inputs, and outputs that are of interest. It differentiates convergent from nonconvergent behaviors using Convergence Classifier Functions (CCFs).\n          <\/jats:p>\n          <jats:p>The tool can be applied in the context of testing models created late in the controller development stage, where it assumes that the given model displays mostly convergent behavior and learns a CCF in an unsupervised fashion from such convergent model behaviors. This CCF is then used to guide a thorough exploration of the model with the help of optimization-guided techniques or adaptive sampling techniques, with the goal of identifying rare nonconvergent model behaviors. Underminer can also be used early in the development stage, where models may have some significant nonconvergent behaviors. Here, the framework permits designers to indicate their mental model for convergence by labeling behaviors as convergent\/nonconvergent and then constructs a CCF using a supervised learning technique. In this use case, the goal is to use the CCF to test an improved design for the model. Underminer supports a number of convergence-like notions, such as those based on Lyapunov analysis and temporal logic, and also CCFs learned directly from labeled output behaviors using machine-learning techniques such as support vector machines and neural networks. We demonstrate the efficacy of Underminer by evaluating its performance on several academic as well as industrial examples.<\/jats:p>","DOI":"10.1145\/3122787","type":"journal-article","created":{"date-parts":[[2017,12,6]],"date-time":"2017-12-06T21:23:15Z","timestamp":1512595395000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Underminer"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0899-659X","authenticated-orcid":false,"given":"Ayca","family":"Balkan","sequence":"first","affiliation":[{"name":"University of California, Los Angeles, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paulo","family":"Tabuada","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jyotirmoy V.","family":"Deshmukh","sequence":"additional","affiliation":[{"name":"Toyota Technical Center, Los Angeles CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaoqing","family":"Jin","sequence":"additional","affiliation":[{"name":"Toyota Technical Center"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James","family":"Kapinski","sequence":"additional","affiliation":[{"name":"Toyota Technical Center"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,12,6]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Y. S. R. Annapureddy C. Liu G. E. Fainekos and S. Sankaranarayanan. 2011. S-TaLiRo: A tool for temporal logic falsification for hybrid systems. In TACAS. 254--257.   Y. S. R. Annapureddy C. Liu G. E. Fainekos and S. Sankaranarayanan. 2011. S-TaLiRo: A tool for temporal logic falsification for hybrid systems. In TACAS. 254--257.","DOI":"10.1007\/978-3-642-19835-9_21"},{"key":"e_1_2_1_2_1","unstructured":"A. Balkan. 2017. Labeling for Not Stable System. Retrieved from http:\/\/www.cyphylab.ee.ucla.edu\/labelingnotstablesystem\/.  A. Balkan. 2017. Labeling for Not Stable System. Retrieved from http:\/\/www.cyphylab.ee.ucla.edu\/labelingnotstablesystem\/."},{"volume-title":"Proc. of the 1st Indian Control Conference. 71--75","author":"Balkan A.","key":"e_1_2_1_3_1"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2968478.2968487"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2728606.2728631"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022627411411"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10703-009-0066-0"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14295-6_17"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"A. Donz\u00e9 and O. Maler. 2010. Robust satisfaction of temporal logic over real-valued signals. In FORMATS. 92--106.   A. Donz\u00e9 and O. Maler. 2010. Robust satisfaction of temporal logic over real-valued signals. In FORMATS. 92--106.","DOI":"10.1007\/978-3-642-15297-9_9"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"T. Dreossi T. Dang A. Donz\u00e9 J. Kapinski X. Jin and J. V. Deshmukh. 2015. Efficient guiding strategies for testing of temporal properties of hybrid systems. In NASA Formal Methods. 127--142.  T. Dreossi T. Dang A. Donz\u00e9 J. Kapinski X. Jin and J. V. Deshmukh. 2015. Efficient guiding strategies for testing of temporal properties of hybrid systems. In NASA Formal Methods. 127--142.","DOI":"10.1007\/978-3-319-17524-9_10"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tcs.2009.06.021"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysconle.2014.05.007"},{"key":"e_1_2_1_13_1","volume-title":"Neural Networks: A Comprehensive Foundation","author":"Haykin S.","year":"1998","edition":"2"},{"key":"e_1_2_1_14_1","first-page":"1","article-title":"Rate-based model predictive controller for diesel engine air path: Design and experimental evaluation","volume":"99","author":"Huang M.","year":"2016","journal-title":"IEEE Trans. Control Syst. Technol."},{"volume-title":"ILOG CPLEX: High-performance software for mathematical programming and optimization.","year":"2006","author":"ILOG.","key":"e_1_2_1_15_1"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2562059.2562140"},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"A. Jones Z. Kong and C. Belta. 2014. Anomaly detection in cyber-physical systems: A formal methods approach. In CDC. 848--853.  A. Jones Z. Kong and C. Belta. 2014. Anomaly detection in cyber-physical systems: A formal methods approach. In CDC. 848--853.","DOI":"10.1109\/CDC.2014.7039487"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2562059.2562139"},{"volume-title":"Nonlinear Systems","author":"Khalil H. K.","key":"e_1_2_1_19_1"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2562059.2562146"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2883817.2883846"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"V. Lakshmikantham S. Leela and A. A. Martynyuk. 1990. Practical Stability of Nonlinear Systems. World Scientific.  V. Lakshmikantham S. Leela and A. A. Martynyuk. 1990. Practical Stability of Nonlinear Systems. World Scientific.","DOI":"10.1142\/1192"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1287\/moor.8.4.538"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CACSD.2004.1393890"},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"O. Maler and D. Nickovic. 2004. Monitoring temporal properties of continuous signals. In FORMATS. 152--166.  O. Maler and D. Nickovic. 2004. Monitoring temporal properties of continuous signals. In FORMATS. 152--166.","DOI":"10.1007\/978-3-540-30206-3_12"},{"key":"e_1_2_1_26_1","unstructured":"Mathworks. 2007. Using Simulink. MathWorks.  Mathworks. 2007. Using Simulink. MathWorks."},{"volume-title":"International Conference on Embedded Software (EMSOFT\u201915)","author":"Medhat R.","key":"e_1_2_1_27_1"},{"key":"e_1_2_1_28_1","unstructured":"B. Messner and D. Tilbury. Control Tutorials for MATLAB and Simulink. Retrieved from http:\/\/ctms.engin.umich.edu\/.  B. Messner and D. Tilbury. Control Tutorials for MATLAB and Simulink. Retrieved from http:\/\/ctms.engin.umich.edu\/."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/7.4.308"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-3108-8"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556789908805762"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2008.03.010"},{"key":"e_1_2_1_33_1","unstructured":"J. Wiens E. Horvitz and J. V. Guttag. 2012. Patient risk stratification for hospital-associated C. diff as a time-series classification task. In Advances in Neural Information Processing Systems 25 F. Pereira C. J. C. Burges L. Bottou and K. Q. Weinberger (Eds.). Curran Associates 467--475. Retrieved from http:\/\/papers.nips.cc\/paper\/4525-patient-risk-stratification-for-hospital-associated-c-diff-as-a-time-series-classification-task.pdf.   J. Wiens E. Horvitz and J. V. Guttag. 2012. Patient risk stratification for hospital-associated C. diff as a time-series classification task. In Advances in Neural Information Processing Systems 25 F. Pereira C. J. C. Burges L. Bottou and K. Q. Weinberger (Eds.). Curran Associates 467--475. Retrieved from http:\/\/papers.nips.cc\/paper\/4525-patient-risk-stratification-for-hospital-associated-c-diff-as-a-time-series-classification-task.pdf."}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3122787","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3122787","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:05:08Z","timestamp":1750273508000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3122787"}},"subtitle":["A Framework for Automatically Identifying Nonconverging Behaviors in Black-Box System Models"],"short-title":[],"issued":{"date-parts":[[2017,12,6]]},"references-count":33,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,1,31]]}},"alternative-id":["10.1145\/3122787"],"URL":"https:\/\/doi.org\/10.1145\/3122787","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"type":"print","value":"1539-9087"},{"type":"electronic","value":"1558-3465"}],"subject":[],"published":{"date-parts":[[2017,12,6]]},"assertion":[{"value":"2017-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-12-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}