{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T19:43:06Z","timestamp":1776109386623,"version":"3.50.1"},"publisher-location":"Cham","reference-count":33,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783031656293","type":"print"},{"value":"9783031656309","type":"electronic"}],"license":[{"start":{"date-parts":[[2024,1,1]],"date-time":"2024-01-01T00:00:00Z","timestamp":1704067200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T00:00:00Z","timestamp":1721865600000},"content-version":"vor","delay-in-days":206,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We present <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\textsf{soid}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>soid<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, a tool for interrogating the decision making of autonomous agents using SMT-based automated reasoning. Relying on the Z3 SMT solver and KLEE symbolic execution engine, <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\textsf{soid}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>soid<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> allows investigators to receive rigorously proven answers to factual and counterfactual queries about agent behavior, enabling effective legal and engineering accountability for harmful or otherwise incorrect decisions. We evaluate <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\textsf{soid}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>soid<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> qualitatively and quantitatively on a pair of examples, i) a buggy implementation of a classic decision tree inference benchmark from the explainable AI (XAI) literature; and ii) a car crash in a simulated physics environment. For the latter, we also contribute the <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\textsf{soid}\\hbox {-}\\!\\textsf{gui}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mi>soid<\/mml:mi>\n                    <mml:mtext>-<\/mml:mtext>\n                    <mml:mspace\/>\n                    <mml:mi>gui<\/mml:mi>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, a domain-specific, web-based example interface for legal and other practitioners to specify factual and counterfactual queries without requiring sophisticated programming or formal methods expertise.<\/jats:p>","DOI":"10.1007\/978-3-031-65630-9_12","type":"book-chapter","created":{"date-parts":[[2024,7,24]],"date-time":"2024-07-24T22:01:56Z","timestamp":1721858516000},"page":"233-246","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["soid: A Tool for\u00a0Legal Accountability for\u00a0Automated Decision Making"],"prefix":"10.1007","author":[{"given":"Samuel","family":"Judson","sequence":"first","affiliation":[]},{"given":"Matthew","family":"Elacqua","sequence":"additional","affiliation":[]},{"given":"Filip","family":"Cano","sequence":"additional","affiliation":[]},{"given":"Timos","family":"Antonopoulos","sequence":"additional","affiliation":[]},{"given":"Bettina","family":"K\u00f6nighofer","sequence":"additional","affiliation":[]},{"given":"Scott J.","family":"Shapiro","sequence":"additional","affiliation":[]},{"given":"Ruzica","family":"Piskac","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,7,25]]},"reference":[{"key":"12_CR1","doi-asserted-by":"publisher","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","volume":"6","author":"A Adadi","year":"2018","unstructured":"Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on Explainable Artificial Intelligence (XAI). IEEE Access 6, 52138\u201352160 (2018)","journal-title":"IEEE Access"},{"key":"12_CR2","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1007\/978-3-540-71209-1_12","volume-title":"Tools and Algorithms for the Construction and Analysis of Systems","author":"S Anand","year":"2007","unstructured":"Anand, S., P\u0103s\u0103reanu, C.S., Visser, W.: JPF\u2013SE: a symbolic execution extension to Java PathFinder. In: Grumberg, O., Huth, M. (eds.) Tools and Algorithms for the Construction and Analysis of Systems, pp. 134\u2013138. Springer Berlin Heidelberg, Berlin, Heidelberg (2007). https:\/\/doi.org\/10.1007\/978-3-540-71209-1_12"},{"key":"12_CR3","unstructured":"Angwin, J., Larson, J., Mattu, S., Kirchner, L.: Machine Bias. ProPublica (May 23rd, 2016). https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing"},{"key":"12_CR4","first-page":"6","volume":"2","author":"C Barrett","year":"2021","unstructured":"Barrett, C., Fontaine, P., Tinelli, C.: The SMT-LIB Standard: Version 2, 6 (2021)","journal-title":"The SMT-LIB Standard: Version"},{"key":"12_CR5","unstructured":"Cadar, C., Dunbar, D., Engler, D.R.: KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In: USENIX Symposium on Operating Systems Design and Implementation (OSDI \u201908), pp. 209\u2013224 (2008)"},{"key":"12_CR6","unstructured":"Chevalier-Boisvert, M., Golemo, F., Cao, Y., Mehta, B., Paull, L.: Duckietown Environments for OpenAI Gym. https:\/\/github.com\/duckietown\/gym-duckietown (2018)"},{"key":"12_CR7","doi-asserted-by":"crossref","unstructured":"Chockler, H., Halpern, J.Y.: Responsibility and blame: a structural-model approach. J. Artif. Intell. Res. 22, 93\u2013115 (2004)","DOI":"10.1613\/jair.1391"},{"key":"12_CR8","unstructured":"Dastin, J.: Amazon scraps secret AI recruiting tool that showed bias against women. Reuters (2018). https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight\/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G"},{"key":"12_CR9","doi-asserted-by":"crossref","unstructured":"Dreossi, T., et al.: VerifAI: a toolkit for the formal design and analysis of artificial intelligence-based systems. In: Intentional Conference on Computer Aided Verification (CAV \u201919), pp. 432\u2013442. Springer (2019)","DOI":"10.1007\/978-3-030-25540-4_25"},{"key":"12_CR10","doi-asserted-by":"crossref","unstructured":"Feigenbaum, J., Jaggard, A.D., Wright, R.N.: Accountability in computing: Concepts and Mechanisms. Found. Trends\u00ae Privacy Security 2(4), 247\u2013399 (2020)","DOI":"10.1561\/3300000002"},{"key":"12_CR11","doi-asserted-by":"crossref","unstructured":"Fremont, D.J., Dreossi, T., Ghosh, S., Yue, X., Sangiovanni-Vincentelli, A.L., Seshia, S.A.: Scenic: a language for scenario specification and scene generation. In: ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI \u201919), pp. 63\u201378 (2019)","DOI":"10.1145\/3314221.3314633"},{"key":"12_CR12","unstructured":"Gopinath, D., Wang, K., Zhang, M., Pasareanu, C.S., Khurshid, S.: Symbolic Execution for Deep Neural Networks. arXiv preprint arXiv:1807.10439 (2018)"},{"issue":"5","key":"12_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3236009","volume":"51","author":"R Guidotti","year":"2018","unstructured":"Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. (CSUR) 51(5), 1\u201342 (2018)","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"12_CR14","doi-asserted-by":"crossref","unstructured":"Halpern, J.Y., Pearl, J.: Causes and explanations: a structural-model approach. part i: causes. British J. Philos. Sci. 56(4), 843\u2013887 (2005)","DOI":"10.1093\/bjps\/axi147"},{"key":"12_CR15","doi-asserted-by":"crossref","unstructured":"Halpern, J.Y., Pearl, J.: Causes and explanations: a structural-model approach. part II: explanations. British J. Philos. Sci. 56(4), 889\u2013911 (2005)","DOI":"10.1093\/bjps\/axi148"},{"key":"12_CR16","doi-asserted-by":"crossref","unstructured":"Judson, S., Elacqua, M., C\u00f3rdoba, F.C., Antonopoulos, T., K\u00f6nighofer, B., Shapiro, S.J., Piskac, R.: \u2018Put the Car on the Stand\u2019: SMT-based Oracles for Investigating Decisions. In: ACM Symposium on Computer Science and Law (CSLAW \u201924) (2024). https:\/\/arxiv.org\/abs\/2305.05731 for an extended technical report","DOI":"10.1145\/3614407.3643699"},{"key":"12_CR17","doi-asserted-by":"crossref","unstructured":"Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: International Conference on Computer Aided Verification (CAV \u201917), pp. 97\u2013117 (2017)","DOI":"10.1007\/978-3-319-63387-9_5"},{"key":"12_CR18","doi-asserted-by":"crossref","unstructured":"Katz, G.: The marabou framework for verification and analysis of deep neural networks. In: International Conference on Computer Aided Verification (CAV \u201919), pp. 443\u2013452 (2019)","DOI":"10.1007\/978-3-030-25540-4_26"},{"issue":"3","key":"12_CR19","first-page":"633","volume":"165","author":"JA Kroll","year":"2017","unstructured":"Kroll, J.A., et al.: Accountable algorithms. Univ. Pa. Law Rev. 165(3), 633\u2013705 (2017)","journal-title":"Univ. Pa. Law Rev."},{"key":"12_CR20","unstructured":"Lewis, D.: Counterfactuals. John Wiley & Sons (2013). originally published in 1973"},{"key":"12_CR21","doi-asserted-by":"crossref","unstructured":"Liew, D., Schemmel, D., Cadar, C., Donaldson, A.F., Zahl, R., Wehrle, K.: Floating-point symbolic execution: a case study in N-version programming. In: IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201917), pp. 601\u2013612 (2017)","DOI":"10.1109\/ASE.2017.8115670"},{"key":"12_CR22","doi-asserted-by":"crossref","unstructured":"Lindner, M., Aparicius, J., Lindgren, P.: No panic! verification of rust programs by symbolic execution. In: 2018 IEEE 16th International Conference on Industrial Informatics (INDIN), pp. 108\u2013114. IEEE (2018)","DOI":"10.1109\/INDIN.2018.8471992"},{"key":"12_CR23","doi-asserted-by":"crossref","unstructured":"Loring, B., Mitchell, D., Kinder, J.: ExpoSE: practical symbolic execution of standalone JavaScript. In: International SPIN Symposium on Model Checking of Software (SPIN \u201917), pp. 196\u2013199 (2017)","DOI":"10.1145\/3092282.3092295"},{"key":"12_CR24","doi-asserted-by":"crossref","unstructured":"Mothilal, R.K., Sharma, A., Tan, C.: Explaining machine learning classifiers through diverse counterfactual explanations. In: ACM Conference on Fairness, Accountability, and Transparency ($$\\text{FAT}^*$$ \u201920), pp. 607\u2013617 (2020)","DOI":"10.1145\/3351095.3372850"},{"issue":"2","key":"12_CR25","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.entcs.2008.04.079","volume":"198","author":"L de Moura","year":"2008","unstructured":"de Moura, L., Bj\u00f8rner, N.: Model-based theory combination. Electron. Notes Theor. Comput. Sci. 198(2), 37\u201349 (2008)","journal-title":"Electron. Notes Theor. Comput. Sci."},{"key":"12_CR26","doi-asserted-by":"crossref","unstructured":"Moura, L.d., Bj\u00f8rner, N.: Z3: An efficient SMT Solver. In: International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS \u201908), pp. 337\u2013340 (2008)","DOI":"10.1007\/978-3-540-78800-3_24"},{"key":"12_CR27","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011)","journal-title":"J. Mach. Learn. Res."},{"key":"12_CR28","unstructured":"Shalev-Shwartz, S., Shammah, S., Shashua, A.: On a Formal Model of Safe and Scalable Self-Driving Cars. arXiv preprint arXiv:1708.06374 (2017)"},{"key":"12_CR29","doi-asserted-by":"crossref","unstructured":"Shoshitaishvili, Y., et al.: SoK: (State of) The art of war: offensive techniques in binary analysis. In: IEEE Symposium on Security and Privacy (S&P \u201916) (2016)","DOI":"10.1109\/SP.2016.17"},{"key":"12_CR30","unstructured":"Smiley, L.: \u2018I\u2019m the Operator\u2019: The Aftermath of a Self-Driving Tragedy. Wired Magazine (2022). https:\/\/www.wired.com\/story\/uber-self-driving-car-fatal-crash\/"},{"key":"12_CR31","first-page":"841","volume":"31","author":"S Wachter","year":"2017","unstructured":"Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harvard J. Law Technolo. 31, 841 (2017)","journal-title":"Harvard J. Law Technolo."},{"key":"12_CR32","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJCH Watkins","year":"1992","unstructured":"Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8, 279\u2013292 (1992)","journal-title":"Mach. Learn."},{"key":"12_CR33","doi-asserted-by":"crossref","unstructured":"Yoon, M.K., Shao, Z.: ADLP: accountable data logging protocol for publish-subscribe communication systems. In: International Conference on Distributed Computing Systems (ICDCS \u201919), pp. 1149\u20131160. IEEE (2019)","DOI":"10.1109\/ICDCS.2019.00117"}],"container-title":["Lecture Notes in Computer Science","Computer Aided Verification"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-65630-9_12","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,24]],"date-time":"2024-07-24T22:03:29Z","timestamp":1721858609000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-65630-9_12"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"ISBN":["9783031656293","9783031656309"],"references-count":33,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-65630-9_12","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024]]},"assertion":[{"value":"25 July 2024","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"CAV","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Computer Aided Verification","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Montreal, QC","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Canada","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2024","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"24 July 2024","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"27 July 2024","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"36","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"cav2024","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"http:\/\/i-cav.org\/2024\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}