{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T09:13:01Z","timestamp":1787908381125,"version":"build-2784847793"},"publisher-location":"New York, NY, USA","reference-count":42,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,11,1]],"date-time":"2022-11-01T00:00:00Z","timestamp":1667260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000870","name":"John D. and Catherine T. MacArthur Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000870","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,11]]},"DOI":"10.1145\/3511265.3550446","type":"proceedings-article","created":{"date-parts":[[2022,11,1]],"date-time":"2022-11-01T04:06:40Z","timestamp":1667275600000},"page":"1-8","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Non-Determinism and the Lawlessness of Machine Learning Code"],"prefix":"10.1145","author":[{"given":"A. Feder","family":"Cooper","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan","family":"Frankle","sequence":"additional","affiliation":[{"name":"MIT & MosaicML, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"De Sa","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,11]]},"reference":[{"key":"e_1_3_2_2_1_1","first-page":"45","article-title":"The Path of Robotics Law","volume":"6","author":"Balkin Jack M.","year":"2015","unstructured":"Jack M. Balkin . 2015 . The Path of Robotics Law . California Law Review Circuit 6 (2015), 45 -- 60 . Jack M. Balkin. 2015. The Path of Robotics Law. California Law Review Circuit 6 (2015), 45--60.","journal-title":"California Law Review Circuit"},{"key":"e_1_3_2_2_2_1","first-page":"2337","article-title":"When a Small Change Makes a Big Difference: Algorithmic Fairness Among Similar Individuals","volume":"55","author":"Bambauer Jane R.","year":"2022","unstructured":"Jane R. Bambauer , Tal Zarsky , and Jonathan Mayer . 2022 . When a Small Change Makes a Big Difference: Algorithmic Fairness Among Similar Individuals . UC Davis Law Review 55 (2022), 2337 -- 2419 . Jane R. Bambauer, Tal Zarsky, and Jonathan Mayer. 2022. When a Small Change Makes a Big Difference: Algorithmic Fairness Among Similar Individuals. UC Davis Law Review 55 (2022), 2337--2419.","journal-title":"UC Davis Law Review"},{"key":"e_1_3_2_2_3_1","first-page":"669","article-title":"Technologies of Compliance: Risk and Regulation in a Digital Age","volume":"88","author":"Bamberger Kenneth A.","year":"2010","unstructured":"Kenneth A. Bamberger . 2010 . Technologies of Compliance: Risk and Regulation in a Digital Age . Texas Law Review 88 , 4 (2010), 669 -- 740 . Kenneth A. Bamberger. 2010. Technologies of Compliance: Risk and Regulation in a Digital Age. Texas Law Review 88, 4 (2010), 669--740.","journal-title":"Texas Law Review"},{"key":"e_1_3_2_2_4_1","first-page":"671","article-title":"Big Data's Disparate Impact","volume":"104","author":"Barocas Solon","year":"2016","unstructured":"Solon Barocas and Andrew D. Selbst . 2016 . Big Data's Disparate Impact . California Law Review 104 , 3 (2016), 671 -- 732 . Solon Barocas and AndrewD. Selbst. 2016. Big Data's Disparate Impact. California Law Review 104, 3 (2016), 671--732.","journal-title":"California Law Review"},{"key":"e_1_3_2_2_5_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"734","author":"Bouthillier Xavier","year":"2019","unstructured":"Xavier Bouthillier , C\u00e9sar Laurent , and Pascal Vincent . 2019 . Unreproducible Research is Reproducible . In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research , Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 725-- 734 . Xavier Bouthillier, C\u00e9sar Laurent, and Pascal Vincent. 2019. Unreproducible Research is Reproducible. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 725--734."},{"key":"e_1_3_2_2_6_1","first-page":"1249","article-title":"Plausible Cuase\": Explanatory Standards in the Age of Powerful Machines","volume":"70","author":"Brennan-Marquez Kiel","year":"2019","unstructured":"Kiel Brennan-Marquez . 2019 . \" Plausible Cuase\": Explanatory Standards in the Age of Powerful Machines . Vanderbilt Law Review 70 (2019), 1249 -- 1301 . Issue 4. Kiel Brennan-Marquez. 2019. \"Plausible Cuase\": Explanatory Standards in the Age of Powerful Machines. Vanderbilt Law Review 70 (2019), 1249--1301. Issue 4.","journal-title":"Vanderbilt Law Review"},{"key":"e_1_3_2_2_7_1","first-page":"513","article-title":"Robotics and the Lessons of Cyberlaw","volume":"103","author":"Calo Ryan","year":"2015","unstructured":"Ryan Calo . 2015 . Robotics and the Lessons of Cyberlaw . California Law Review 103 , 3 (2015), 513 -- 563 . Ryan Calo. 2015. Robotics and the Lessons of Cyberlaw. California Law Review 103, 3 (2015), 513--563.","journal-title":"California Law Review"},{"key":"e_1_3_2_2_8_1","article-title":"Modeling Through","volume":"72","author":"Calo Ryan","year":"2021","unstructured":"Ryan Calo . 2021 . Modeling Through . Duke Law Journal 72 (2021), 28 pages. Ryan Calo. 2021. Modeling Through. Duke Law Journal 72 (2021), 28 pages.","journal-title":"Duke Law Journal"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"crossref","unstructured":"Anthony Casey and Anthony Niblett. 2015. The Death of Rules and Standards. 56 pages. Coase-Sandor Working Paper Series in Law and Economics.  Anthony Casey and Anthony Niblett. 2015. The Death of Rules and Standards. 56 pages. Coase-Sandor Working Paper Series in Law and Economics.","DOI":"10.2139\/ssrn.2693826"},{"key":"e_1_3_2_2_10_1","first-page":"1249","article-title":"Technological Due Process","volume":"85","author":"Citron Danielle Keats","year":"2008","unstructured":"Danielle Keats Citron . 2008 . Technological Due Process . Washington University Law Review 85 , 6 (2008), 1249 -- 1314 . Danielle Keats Citron. 2008. Technological Due Process. Washington University Law Review 85, 6 (2008), 1249--1314.","journal-title":"Washington University Law Review"},{"key":"e_1_3_2_2_11_1","first-page":"655","article-title":"The Scored Society: Due Process for Automated Predictions","volume":"89","author":"Citron Danielle Keats","year":"2014","unstructured":"Danielle Keats Citron and Frank A. Pasquale . 2014 . The Scored Society: Due Process for Automated Predictions . Washington Law Review 89 (2014), 655 -- 690 . Danielle Keats Citron and Frank A. Pasquale. 2014. The Scored Society: Due Process for Automated Predictions. Washington Law Review 89 (2014), 655--690.","journal-title":"Washington Law Review"},{"key":"e_1_3_2_2_12_1","article-title":"Privacy Harms","volume":"102","author":"Citron Danielle Keats","year":"2022","unstructured":"Danielle Keats Citron and Daniel J. Solove . 2022 . Privacy Harms . Boston University Law Review 102 (2022), 62 pages. Danielle Keats Citron and Daniel J. Solove. 2022. Privacy Harms. Boston University Law Review 102 (2022), 62 pages.","journal-title":"Boston University Law Review"},{"key":"e_1_3_2_2_13_1","volume-title":"Fast or Accurate? Governing Conflicting Goals in Highly Autonomous Vehicles. Colorado Technology Law Journal 20","author":"Feder Cooper A.","year":"2022","unstructured":"A. Feder Cooper and Karen Levy . 2022. Fast or Accurate? Governing Conflicting Goals in Highly Autonomous Vehicles. Colorado Technology Law Journal 20 ( 2022 ). A. Feder Cooper and Karen Levy. 2022. Fast or Accurate? Governing Conflicting Goals in Highly Autonomous Vehicles. Colorado Technology Law Journal 20 (2022)."},{"key":"e_1_3_2_2_14_1","volume-title":"Equity and Access in Algorithms, Mechanisms, and Optimization","author":"Cooper A. Feder","unstructured":"A. Feder Cooper , Karen Levy , and Christopher De Sa. 2021. Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems . In Equity and Access in Algorithms, Mechanisms, and Optimization . Association for Computing Machinery , New York, NY, USA , Article 4, 11 pages. A. Feder Cooper, Karen Levy, and Christopher De Sa. 2021. Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems. In Equity and Access in Algorithms, Mechanisms, and Optimization. Association for Computing Machinery, New York, NY, USA, Article 4, 11 pages."},{"key":"e_1_3_2_2_15_1","volume-title":"Advances in Neural Information Processing Systems","volume":"34","author":"Cooper A. Feder","year":"2021","unstructured":"A. Feder Cooper , Yucheng Lu , Jessica Zosa Forde , and Christopher De Sa . 2021 . Hyperparameter Optimization Is Deceiving Us, and How to Stop It . In Advances in Neural Information Processing Systems , Vol. 34 . Curran Associates, Inc., 43 pages. A. Feder Cooper, Yucheng Lu, Jessica Zosa Forde, and Christopher De Sa. 2021. Hyperparameter Optimization Is Deceiving Us, and How to Stop It. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., 43 pages."},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533150"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1017\/can.2022.3"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3512898"},{"key":"e_1_3_2_2_19_1","first-page":"759","article-title":"Governing Online Speech: From 'Posts-As-Trumps' to Proportionality and Probability","volume":"121","year":"2021","unstructured":"evelyn douek. 2021 . Governing Online Speech: From 'Posts-As-Trumps' to Proportionality and Probability . Columbia Law Review 121 , 3 (2021), 759 -- 834 . evelyn douek. 2021. Governing Online Speech: From 'Posts-As-Trumps' to Proportionality and Probability. Columbia Law Review 121, 3 (2021), 759--834.","journal-title":"Columbia Law Review"},{"key":"e_1_3_2_2_20_1","volume-title":"The Siren Call of Content Moderation Formalism. New Technologies of Communication and the First Amendment: The Internet, Social Media and Censorship","year":"2022","unstructured":"evelyn douek. 2022. The Siren Call of Content Moderation Formalism. New Technologies of Communication and the First Amendment: The Internet, Social Media and Censorship ( 2022 ). Forthcoming . evelyn douek. 2022. The Siren Call of Content Moderation Formalism. New Technologies of Communication and the First Amendment: The Internet, Social Media and Censorship (2022). Forthcoming."},{"key":"e_1_3_2_2_21_1","article-title":"The Impact of Artificial Intelligence on Rules, Standards, and Judicial Discretion","volume":"93","author":"Fagan Frank","year":"2019","unstructured":"Frank Fagan and Saul Levmore . 2019 . The Impact of Artificial Intelligence on Rules, Standards, and Judicial Discretion . Southern California Law Review 93 , 1 (2019), 35 pages. Frank Fagan and Saul Levmore. 2019. The Impact of Artificial Intelligence on Rules, Standards, and Judicial Discretion. Southern California Law Review 93, 1 (2019), 35 pages.","journal-title":"Southern California Law Review"},{"key":"e_1_3_2_2_22_1","volume-title":"Chris De Sa, and Michael L. Littman","author":"Forde Jessica Zosa","year":"2021","unstructured":"Jessica Zosa Forde , A. Feder Cooper , Kweku Kwegyir-Aggrey , Chris De Sa, and Michael L. Littman . 2021 . Model Selection's Disparate Impact in Real-World Deep Learning Applications . https:\/\/arxiv.org\/abs\/2104.00606 Jessica Zosa Forde, A. Feder Cooper, Kweku Kwegyir-Aggrey, Chris De Sa, and Michael L. Littman. 2021. Model Selection's Disparate Impact in Real-World Deep Learning Applications. https:\/\/arxiv.org\/abs\/2104.00606"},{"key":"e_1_3_2_2_23_1","volume-title":"The Morality of Law","author":"Fuller Lon L.","unstructured":"Lon L. Fuller . 1965. The Morality of Law . Yale University Press , New Haven . Lon L. Fuller. 1965. The Morality of Law. Yale University Press, New Haven."},{"key":"e_1_3_2_2_24_1","first-page":"1719","article-title":"Regulation by Software","volume":"114","author":"Grimmelmann James","year":"2005","unstructured":"James Grimmelmann . 2005 . Regulation by Software . The Yale Law Journal 114 (2005), 1719 -- 1758 . Journal Note. James Grimmelmann. 2005. Regulation by Software. The Yale Law Journal 114 (2005), 1719--1758. Journal Note.","journal-title":"The Yale Law Journal"},{"key":"e_1_3_2_2_26_1","first-page":"655","article-title":"Smooth and Bumpy Laws","volume":"102","author":"Kolber Adam J.","year":"2014","unstructured":"Adam J. Kolber . 2014 . Smooth and Bumpy Laws . California Law Review 102 (2014), 655 -- 690 . Adam J. Kolber. 2014. Smooth and Bumpy Laws. California Law Review 102 (2014), 655--690.","journal-title":"California Law Review"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/0022-0000(81)90036-2"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/800061.808758"},{"key":"e_1_3_2_2_29_1","first-page":"633","article-title":"Accountable Algorithms","volume":"165","author":"Kroll Joshua A.","year":"2017","unstructured":"Joshua A. Kroll , Joanna Huey , Solon Barocas , Edward W. Felten , Joel R. Reidenberg , David G. Robinson , and Harlan Yu . 2017 . Accountable Algorithms . University of Pennsylvania Law Review 165 (2017), 633 -- 705 . Issue 633. Joshua A. Kroll, Joanna Huey, Solon Barocas, EdwardW. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu. 2017. Accountable Algorithms. University of Pennsylvania Law Review 165 (2017), 633--705. Issue 633.","journal-title":"University of Pennsylvania Law Review"},{"key":"e_1_3_2_2_30_1","first-page":"653","article-title":"Playing with the Data: What Legal Scholars Should Learn About Machine Learning","volume":"51","author":"Lehr David","year":"2017","unstructured":"David Lehr and Paul Ohm . 2017 . Playing with the Data: What Legal Scholars Should Learn About Machine Learning . U.C. Davis Law Review 51 (2017), 653 -- 717 . David Lehr and Paul Ohm. 2017. Playing with the Data: What Legal Scholars Should Learn About Machine Learning. U.C. Davis Law Review 51 (2017), 653--717.","journal-title":"U.C. Davis Law Review"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.2307\/1342331"},{"key":"e_1_3_2_2_32_1","first-page":"1","article-title":"Law Regulating Code Regulating Law","volume":"35","author":"Lessig Lawrence","year":"2003","unstructured":"Lawrence Lessig . 2003 . Law Regulating Code Regulating Law . Loyal University Chicago Law Journal 35 , 1 (2003), 1 -- 14 . Lawrence Lessig. 2003. Law Regulating Code Regulating Law. Loyal University Chicago Law Journal 35, 1 (2003), 1--14.","journal-title":"Loyal University Chicago Law Journal"},{"key":"e_1_3_2_2_33_1","volume-title":"Lawrence Lessig. 2009. Code 2.0","unstructured":"Lawrence Lessig. 2009. Code 2.0 ( 2 nd ed.). CreateSpace , Scotts Valley, CA . Lawrence Lessig. 2009. Code 2.0 (2nd ed.). CreateSpace, Scotts Valley, CA.","edition":"2"},{"key":"e_1_3_2_2_34_1","unstructured":"D.K. Mulligan and K.A. Bamberger. 2018. Saving governance-by-design. California Law Review 106 (June 2018) 697--784.  D.K. Mulligan and K.A. Bamberger. 2018. Saving governance-by-design. California Law Review 106 (June 2018) 697--784."},{"key":"e_1_3_2_2_35_1","volume-title":"Bamberger","author":"Mulligan Deirdre K.","year":"2019","unstructured":"Deirdre K. Mulligan and Kenneth A . Bamberger . 2019 . Procurement As Policy: Administrative Process for Machine Learning. Berkeley Technology Law Journal 34 (4 Oct. 2019), 771--858. Deirdre K. Mulligan and Kenneth A. Bamberger. 2019. Procurement As Policy: Administrative Process for Machine Learning. Berkeley Technology Law Journal 34 (4 Oct. 2019), 771--858."},{"key":"e_1_3_2_2_36_1","volume-title":"Probabilistic Machine Learning: An introduction","author":"Murphy Kevin P.","unstructured":"Kevin P. Murphy . 2022. Probabilistic Machine Learning: An introduction . MIT Press . probml.ai Kevin P. Murphy. 2022. Probabilistic Machine Learning: An introduction. MIT Press. probml.ai"},{"key":"e_1_3_2_2_37_1","volume-title":"An Empirical Study of Fixed-Seed Training. In Advances in Neural Information Processing Systems","volume":"34","author":"Qian Shangshu","year":"2021","unstructured":"Shangshu Qian , Hung Pham , Thibaud Lutellier , Zeou Hu , Jungwon Kim , Lin Tan , Yaoliang Yu , Jiahao Chen , and Sameena Shah . 2021 . Are My Deep Learning Systems Fair ? An Empirical Study of Fixed-Seed Training. In Advances in Neural Information Processing Systems , Vol. 34 . Curran Associates, Inc., 17 pages. Shangshu Qian, Hung Pham, Thibaud Lutellier, Zeou Hu, Jungwon Kim, Lin Tan, Yaoliang Yu, Jiahao Chen, and Sameena Shah. 2021. Are My Deep Learning Systems Fair? An Empirical Study of Fixed-Seed Training. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., 17 pages."},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454779"},{"key":"e_1_3_2_2_39_1","first-page":"553","article-title":"Lex Informatica: The Formulation of Information Policy Rules through Technology","volume":"76","author":"Reidenberg Joel R.","year":"1997","unstructured":"Joel R. Reidenberg . 1997 . Lex Informatica: The Formulation of Information Policy Rules through Technology . Texas Law Review 76 , 3 (1997), 553 -- 594 . Joel R. Reidenberg. 1997. Lex Informatica: The Formulation of Information Policy Rules through Technology. Texas Law Review 76, 3 (1997), 553--594.","journal-title":"Texas Law Review"},{"key":"e_1_3_2_2_40_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research","author":"Sivaprasad Prabhu Teja","year":"2020","unstructured":"Prabhu Teja Sivaprasad , Florian Mai , Thijs Vogels , Martin Jaggi , and Fran\u00e7ois Fleuret . 2020 . Optimizer Benchmarking Needs to Account for Hyperparameter Tuning . In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research , Vol. 119), Hal Daum\u00e9 III and Aarti Singh (Eds.). PMLR, 9036--9045. Prabhu Teja Sivaprasad, Florian Mai, Thijs Vogels, Martin Jaggi, and Fran\u00e7ois Fleuret. 2020. Optimizer Benchmarking Needs to Account for Hyperparameter Tuning. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daum\u00e9 III and Aarti Singh (Eds.). PMLR, 9036--9045."},{"key":"e_1_3_2_2_41_1","volume-title":"Decision Time: Normative Dimensions of Algorithmic Speed. In 2022 ACM Conference on Fairness, Accountability, and Transparency","author":"Susser Daniel","year":"2022","unstructured":"Daniel Susser . 2022 . Decision Time: Normative Dimensions of Algorithmic Speed. In 2022 ACM Conference on Fairness, Accountability, and Transparency ( Seoul, Republic of Korea) (FAccT '22). Association for Computing Machinery, New York, NY, USA, 1410--1420. Daniel Susser. 2022. Decision Time: Normative Dimensions of Algorithmic Speed. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT '22). Association for Computing Machinery, New York, NY, USA, 1410--1420."},{"key":"e_1_3_2_2_42_1","volume-title":"Politics, Theory","author":"Tamanaha Brian Z.","unstructured":"Brian Z. Tamanaha . 2004. On the Rule of Law: History , Politics, Theory . Cambridge University Press , Cambridge . Brian Z. Tamanaha. 2004. On the Rule of Law: History, Politics, Theory. Cambridge University Press, Cambridge."},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.2307\/1339610"}],"event":{"name":"CSLAW '22: Symposium on Computer Science and Law","location":"Washington DC USA","acronym":"CSLAW '22","sponsor":["ACM Association for Computing Machinery"]},"container-title":["Proceedings of the 2022 Symposium on Computer Science and Law"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511265.3550446","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3511265.3550446","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:59Z","timestamp":1750191119000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511265.3550446"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11]]},"references-count":42,"alternative-id":["10.1145\/3511265.3550446","10.1145\/3511265"],"URL":"https:\/\/doi.org\/10.1145\/3511265.3550446","relation":{},"subject":[],"published":{"date-parts":[[2022,11]]},"assertion":[{"value":"2022-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}