{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T06:30:05Z","timestamp":1750746605074,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":29,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T00:00:00Z","timestamp":1701648000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["OAC-2233871"],"award-info":[{"award-number":["OAC-2233871"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,12,4]]},"DOI":"10.1145\/3617574.3617861","type":"proceedings-article","created":{"date-parts":[[2023,12,1]],"date-time":"2023-12-01T03:26:25Z","timestamp":1701401185000},"page":"18-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Towards Safe ML-Based Systems in Presence of Feedback Loops"],"prefix":"10.1145","author":[{"given":"Sumon","family":"Biswas","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yining","family":"She","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eunsuk","family":"Kang","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,12,4]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"https:\/\/www.cnn.com\/2021\/11\/17\/health\/drug-overdose-deaths-record-high \t\t\t\t  https:\/\/www.cnn.com\/2021\/11\/17\/health\/drug-overdose-deaths-record-high"},{"key":"e_1_3_2_1_2_1","unstructured":"https:\/\/bamboohealth.com\/solutions\/narxcare \t\t\t\t  https:\/\/bamboohealth.com\/solutions\/narxcare"},{"key":"e_1_3_2_1_3_1","unstructured":"https:\/\/www.in.gov\/pla\/inspect\/files\/Narxcare_user_guide.pdf \t\t\t\t  https:\/\/www.in.gov\/pla\/inspect\/files\/Narxcare_user_guide.pdf"},{"key":"e_1_3_2_1_4_1","unstructured":"https:\/\/www.wired.com\/story\/opioid-drug-addiction-algorithm-chronic-pain \t\t\t\t  https:\/\/www.wired.com\/story\/opioid-drug-addiction-algorithm-chronic-pain"},{"key":"e_1_3_2_1_5_1","unstructured":"Dario Amodei Chris Olah Jacob Steinhardt Paul Christiano John Schulman and Dan Man\u00e9. 2016. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565. \t\t\t\t  Dario Amodei Chris Olah Jacob Steinhardt Paul Christiano John Schulman and Dan Man\u00e9. 2016. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565."},{"volume-title":"Feedback systems: an introduction for scientists and engineers","author":"\u00c5str\u00f6m Karl Johan","key":"e_1_3_2_1_6_1","unstructured":"Karl Johan \u00c5str\u00f6m and Richard M Murray . 2021. Feedback systems: an introduction for scientists and engineers . Princeton university press . Karl Johan \u00c5str\u00f6m and Richard M Murray. 2021. Feedback systems: an introduction for scientists and engineers. Princeton university press."},{"key":"e_1_3_2_1_7_1","unstructured":"Maria Casimiro Paolo Romano David Garlan Gabriel A Moreno Eunsuk Kang and Mark Klein. 2021. Self-Adaptation for Machine Learning Based Systems. \t\t\t\t  Maria Casimiro Paolo Romano David Garlan Gabriel A Moreno Eunsuk Kang and Mark Klein. 2021. Self-Adaptation for Machine Learning Based Systems."},{"key":"e_1_3_2_1_8_1","volume-title":"Locality of Technical Objects and the Role of Structural Interventions for Systemic Change. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 2327\u20132341","author":"Cort\u00e9s Efr\u00e9n Cruz","year":"2022","unstructured":"Efr\u00e9n Cruz Cort\u00e9s , Sarah Rajtmajer , and Debashis Ghosh . 2022 . Locality of Technical Objects and the Role of Structural Interventions for Systemic Change. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 2327\u20132341 . Efr\u00e9n Cruz Cort\u00e9s, Sarah Rajtmajer, and Debashis Ghosh. 2022. Locality of Technical Objects and the Role of Structural Interventions for Systemic Change. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 2327\u20132341."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372878"},{"key":"e_1_3_2_1_10_1","volume-title":"Runaway Feedback Loops in Predictive Policing. In Conference on Fairness, Accountability and Transparency FAT. PMLR, 160\u2013171","author":"Ensign Danielle","year":"2018","unstructured":"Danielle Ensign , Sorelle A. Friedler , Scott Neville , Carlos Scheidegger , and Suresh Venkatasubramanian . 2018 . Runaway Feedback Loops in Predictive Policing. In Conference on Fairness, Accountability and Transparency FAT. PMLR, 160\u2013171 . Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian. 2018. Runaway Feedback Loops in Predictive Policing. In Conference on Fairness, Accountability and Transparency FAT. PMLR, 160\u2013171."},{"key":"e_1_3_2_1_11_1","volume-title":"On Adaptive Fairness in Software Systems. In International Symposium on Software Engineering for Adaptive and Self-Managing Systems.","author":"Farahani Ali","year":"2021","unstructured":"Ali Farahani , Liliana Pasquale , Amel Bennaceur , Thomas Welsh , and Bashar Nuseibeh . 2021 . On Adaptive Fairness in Software Systems. In International Symposium on Software Engineering for Adaptive and Self-Managing Systems. Ali Farahani, Liliana Pasquale, Amel Bennaceur, Thomas Welsh, and Bashar Nuseibeh. 2021. On Adaptive Fairness in Software Systems. In International Symposium on Software Engineering for Adaptive and Self-Managing Systems."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1905355117"},{"volume-title":"Feedback control of computing systems","author":"Hellerstein Joseph L","key":"e_1_3_2_1_13_1","unstructured":"Joseph L Hellerstein , Yixin Diao , Sujay Parekh , and Dawn M Tilbury . 2004. Feedback control of computing systems . John Wiley & Sons . Joseph L Hellerstein, Yixin Diao, Sujay Parekh, and Dawn M Tilbury. 2004. Feedback control of computing systems. John Wiley & Sons."},{"key":"e_1_3_2_1_14_1","volume-title":"Runtime Monitoring of Dynamic Fairness Properties. In ACM Conference on Fairness, Accountability, and Transparency (FAccT). ACM, 604\u2013614","author":"Henzinger Thomas A.","year":"2023","unstructured":"Thomas A. Henzinger , Mahyar Karimi , Konstantin Kueffner , and Kaushik Mallik . 2023 . Runtime Monitoring of Dynamic Fairness Properties. In ACM Conference on Fairness, Accountability, and Transparency (FAccT). ACM, 604\u2013614 . Thomas A. Henzinger, Mahyar Karimi, Konstantin Kueffner, and Kaushik Mallik. 2023. Runtime Monitoring of Dynamic Fairness Properties. In ACM Conference on Fairness, Accountability, and Transparency (FAccT). ACM, 604\u2013614."},{"volume-title":"Problem frames: analysing and structuring software development problems","author":"Jackson Michael","key":"e_1_3_2_1_15_1","unstructured":"Michael Jackson . 2001. Problem frames: analysing and structuring software development problems . Addison-Wesley . Michael Jackson. 2001. Problem frames: analysing and structuring software development problems. Addison-Wesley."},{"key":"e_1_3_2_1_16_1","unstructured":"Max Jaderberg Valentin Dalibard Simon Osindero Wojciech M Czarnecki Jeff Donahue Ali Razavi Oriol Vinyals Tim Green Iain Dunning and Karen Simonyan. 2017. Population based training of neural networks. arXiv preprint arXiv:1711.09846. \t\t\t\t  Max Jaderberg Valentin Dalibard Simon Osindero Wojciech M Czarnecki Jeff Donahue Ali Razavi Oriol Vinyals Tim Green Iain Dunning and Karen Simonyan. 2017. Population based training of neural networks. arXiv preprint arXiv:1711.09846."},{"key":"e_1_3_2_1_17_1","volume-title":"System Safety Engineering for Social and Ethical ML Risks: A Case Study. In NeurIPS ML Safety Workshop.","author":"Jatho Edgar","year":"2022","unstructured":"Edgar Jatho , Logan Mailloux , Shalaleh Rismani , Eugene Williams , and Joshua A Kroll . 2022 . System Safety Engineering for Social and Ethical ML Risks: A Case Study. In NeurIPS ML Safety Workshop. Edgar Jatho, Logan Mailloux, Shalaleh Rismani, Eugene Williams, and Joshua A Kroll. 2022. System Safety Engineering for Social and Ethical ML Risks: A Case Study. In NeurIPS ML Safety Workshop."},{"key":"e_1_3_2_1_18_1","volume-title":"Requirements Engineering for Feedback Loops in Software-Intensive Systems. In 2022 IEEE 30th International Requirements Engineering Conference Workshops (REW). 2\u20135.","author":"Kang Eunsuk","year":"2022","unstructured":"Eunsuk Kang and R\u00f4mulo Meira-G\u00f3es . 2022 . Requirements Engineering for Feedback Loops in Software-Intensive Systems. In 2022 IEEE 30th International Requirements Engineering Conference Workshops (REW). 2\u20135. Eunsuk Kang and R\u00f4mulo Meira-G\u00f3es. 2022. Requirements Engineering for Feedback Loops in Software-Intensive Systems. In 2022 IEEE 30th International Requirements Engineering Conference Workshops (REW). 2\u20135."},{"key":"e_1_3_2_1_19_1","unstructured":"David Krueger Tegan Maharaj and Jan Leike. 2020. Hidden incentives for auto-induced distributional shift. arXiv preprint arXiv:2009.09153. \t\t\t\t  David Krueger Tegan Maharaj and Jan Leike. 2020. Hidden incentives for auto-induced distributional shift. arXiv preprint arXiv:2009.09153."},{"key":"e_1_3_2_1_20_1","unstructured":"Kristian Lum and William Isaac. 2016. To predict and serve? Significance. \t\t\t\t  Kristian Lum and William Isaac. 2016. To predict and serve? Significance."},{"key":"e_1_3_2_1_21_1","volume-title":"Andrew Smart, and William S Isaac","author":"Jr Donald Martin","year":"2020","unstructured":"Donald Martin Jr , Vinodkumar Prabhakaran , Jill Kuhlberg , Andrew Smart, and William S Isaac . 2020 . Extending the machine learning abstraction boundary: A Complex systems approach to incorporate societal context. arXiv preprint arXiv:2006.09663. Donald Martin Jr, Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, and William S Isaac. 2020. Extending the machine learning abstraction boundary: A Complex systems approach to incorporate societal context. arXiv preprint arXiv:2006.09663."},{"key":"e_1_3_2_1_22_1","volume-title":"Business Dynamics: Systems Thinking and Modeling for a Complex World","author":"Meadows Donella H.","year":"2008","unstructured":"Donella H. Meadows . 2008 . Business Dynamics: Systems Thinking and Modeling for a Complex World . Chelsea Green Publishing . Donella H. Meadows. 2008. Business Dynamics: Systems Thinking and Modeling for a Complex World. Chelsea Green Publishing."},{"volume-title":"Advances in Computer Vision and Computational Biology","author":"Mensah Ethel AM","key":"e_1_3_2_1_23_1","unstructured":"Ethel AM Mensah , Musarath J Rahmathullah , Pooja Kumar , Roozbeh Sadeghian , and Siamak Aram . 2021. A Proactive Approach to Combating the Opioid Crisis Using Machine Learning Techniques . In Advances in Computer Vision and Computational Biology . Springer , 385\u2013398. Ethel AM Mensah, Musarath J Rahmathullah, Pooja Kumar, Roozbeh Sadeghian, and Siamak Aram. 2021. A Proactive Approach to Combating the Opioid Crisis Using Machine Learning Techniques. In Advances in Computer Vision and Computational Biology. Springer, 385\u2013398."},{"key":"e_1_3_2_1_24_1","volume-title":"Meta-Learning Update Rules for Unsupervised Representation Learning. In International Conference on Learning Representations.","author":"Metz Luke","year":"2018","unstructured":"Luke Metz , Niru Maheswaranathan , Brian Cheung , and Jascha Sohl-Dickstein . 2018 . Meta-Learning Update Rules for Unsupervised Representation Learning. In International Conference on Learning Representations. Luke Metz, Niru Maheswaranathan, Brian Cheung, and Jascha Sohl-Dickstein. 2018. Meta-Learning Update Rules for Unsupervised Representation Learning. In International Conference on Learning Representations."},{"key":"e_1_3_2_1_25_1","unstructured":"Cathy O\u2019neil. 2017. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown. \t\t\t\t  Cathy O\u2019neil. 2017. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown."},{"volume-title":"Dataset shift in machine learning","author":"Quinonero-Candela Joaquin","key":"e_1_3_2_1_26_1","unstructured":"Joaquin Quinonero-Candela , Masashi Sugiyama , Anton Schwaighofer , and Neil D Lawrence . 2022. Dataset shift in machine learning . Mit Press . Joaquin Quinonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. 2022. Dataset shift in machine learning. Mit Press."},{"key":"e_1_3_2_1_27_1","volume-title":"2022 ACM Conference on Fairness, Accountability, and Transparency","author":"Reader Lydia","year":"2022","unstructured":"Lydia Reader , Pegah Nokhiz , Cathleen Power , Neal Patwari , Suresh Venkatasubramanian , and Sorelle A. Friedler . 2022. Models for understanding and quantifying feedback in societal systems. In FAccT \u201922 : 2022 ACM Conference on Fairness, Accountability, and Transparency , Seoul, Republic of Korea, June 21 - 24 , 2022 . Lydia Reader, Pegah Nokhiz, Cathleen Power, Neal Patwari, Suresh Venkatasubramanian, and Sorelle A. Friedler. 2022. Models for understanding and quantifying feedback in societal systems. In FAccT \u201922: 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea, June 21 - 24, 2022."},{"key":"e_1_3_2_1_28_1","volume-title":"Self-adaptive software: Landscape and research challenges. TAAS, 4, 2","author":"Salehie Mazeiar","year":"2009","unstructured":"Mazeiar Salehie and Ladan Tahvildari . 2009. Self-adaptive software: Landscape and research challenges. TAAS, 4, 2 ( 2009 ), 14:1\u201314:42. Mazeiar Salehie and Ladan Tahvildari. 2009. Self-adaptive software: Landscape and research challenges. TAAS, 4, 2 (2009), 14:1\u201314:42."},{"key":"e_1_3_2_1_29_1","volume-title":"Business Dynamics: Systems Thinking and Modeling for a Complex World","author":"Sterman John D","year":"2000","unstructured":"John D Sterman . 2000 . Business Dynamics: Systems Thinking and Modeling for a Complex World . McGraw-Hill Education . John D Sterman. 2000. Business Dynamics: Systems Thinking and Modeling for a Complex World. McGraw-Hill Education."}],"event":{"name":"SE4SafeML '23: 1st International Workshop on Dependability and Trustworthiness of Safety-Critical Systems with Machine Learned Components","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"],"location":"San Francisco CA USA","acronym":"SE4SafeML '23"},"container-title":["Proceedings of the 1st International Workshop on Dependability and Trustworthiness of Safety-Critical Systems with Machine Learned Components"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617574.3617861","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3617574.3617861","content-type":"text\/html","content-version":"vor","intended-application":"syndication"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:45:59Z","timestamp":1750178759000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617574.3617861"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,4]]},"references-count":29,"alternative-id":["10.1145\/3617574.3617861","10.1145\/3617574"],"URL":"https:\/\/doi.org\/10.1145\/3617574.3617861","relation":{},"subject":[],"published":{"date-parts":[[2023,12,4]]},"assertion":[{"value":"2023-12-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}