{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T05:06:05Z","timestamp":1770267965964,"version":"3.49.0"},"reference-count":32,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,1,24]],"date-time":"2023-01-24T00:00:00Z","timestamp":1674518400000},"content-version":"vor","delay-in-days":23,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,1,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Every legal case sets a precedent by developing the law in one of the following two ways. It either expands its scope, in which case it sets positive precedent, or it narrows it, in which case it sets negative precedent. Legal outcome prediction, the prediction of positive outcome, is an increasingly popular task in AI. In contrast, we turn our focus to negative outcomes here, and introduce a new task of negative outcome prediction. We discover an asymmetry in existing models\u2019 ability to predict positive and negative outcomes. Where the state-of-the-art outcome prediction model we used predicts positive outcomes at 75.06 F1, it predicts negative outcomes at only 10.09 F1, worse than a random baseline. To address this performance gap, we develop two new models inspired by the dynamics of a court process. Our first model significantly improves positive outcome prediction score to 77.15 F1 and our second model more than doubles the negative outcome prediction performance to 24.01 F1. Despite this improvement, shifting focus to negative outcomes reveals that there is still much room for improvement for outcome prediction models.<\/jats:p>\n               <jats:p>https:\/\/github.com\/valvoda\/Negative-Precedent-in-Legal-Outcome-Prediction<\/jats:p>","DOI":"10.1162\/tacl_a_00532","type":"journal-article","created":{"date-parts":[[2023,1,24]],"date-time":"2023-01-24T16:18:28Z","timestamp":1674577108000},"page":"34-48","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":12,"title":["On the Role of Negative Precedent in Legal Outcome Prediction"],"prefix":"10.1162","volume":"11","author":[{"given":"Josef","family":"Valvoda","sequence":"first","affiliation":[{"name":"University of Cambridge, UK. jv406@cam.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ryan","family":"Cotterell","sequence":"additional","affiliation":[{"name":"ETH Z\u00fcrich, Switzerland. ryan.cotterell@inf.ethz.ch"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simone","family":"Teufel","sequence":"additional","affiliation":[{"name":"University of Cambridge, UK. sht25@cam.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,1,12]]},"reference":[{"key":"2023012416102769800_bib1","doi-asserted-by":"publisher","first-page":"e93","DOI":"10.7717\/peerj-cs.93","article-title":"Predicting judicial decisions of the European Court of Human Rights: A natural language processing perspective","volume":"2","author":"Aletras","year":"2016","journal-title":"PeerJ Computer Science"},{"key":"2023012416102769800_bib2","unstructured":"Kevin D.\n              Ashley\n            \n          . 1988. Modelling Legal Argument: Reasoning with Cases and Hypotheticals. Ph.D. thesis, USA. Order No: GAX88-13198."},{"key":"2023012416102769800_bib3","volume-title":"Black\u2019s Law Dictionary","author":"Black","year":"2019","edition":"11th edition"},{"key":"2023012416102769800_bib4","doi-asserted-by":"publisher","first-page":"254","DOI":"10.18653\/v1\/E17-2041","article-title":"Legal NERC with ontologies, Wikipedia and curriculum learning","volume-title":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers","author":"Cardellino","year":"2017"},{"key":"2023012416102769800_bib5","doi-asserted-by":"publisher","first-page":"4317","DOI":"10.18653\/v1\/P19-1424","article-title":"Neural legal judgment prediction in English","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Chalkidis","year":"2019"},{"key":"2023012416102769800_bib6","doi-asserted-by":"publisher","first-page":"2898","DOI":"10.18653\/v1\/2020.findings-emnlp.261","article-title":"LEGAL-BERT: The muppets straight out of law school","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Chalkidis","year":"2020"},{"key":"2023012416102769800_bib7","doi-asserted-by":"publisher","first-page":"226","DOI":"10.18653\/v1\/2021.naacl-main.22","article-title":"Paragraph-level rationale extraction through regularization: A case study on European court of human rights cases","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Chalkidis","year":"2021"},{"key":"2023012416102769800_bib8","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3936759","article-title":"LexGLUE: A benchmark dataset for legal language understanding in English","author":"Chalkidis","year":"2021","journal-title":"arXiv"},{"key":"2023012416102769800_bib9","doi-asserted-by":"publisher","DOI":"10.3233\/FAIA210317","article-title":"The unreasonable effectiveness of the baseline: Discussing SVMs in legal text classification","author":"Clavi\u00e9","year":"2021","journal-title":"arXiv"},{"key":"2023012416102769800_bib10","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2023012416102769800_bib11","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511818684","volume-title":"The Nature and Authority of Precedent","author":"Duxbury","year":"2008"},{"key":"2023012416102769800_bib12","first-page":"89","article-title":"What is law for the European court of human rights","volume":"49","author":"Dzehtsiarou","year":"2017","journal-title":"Georgetown Journal of International Law"},{"key":"2023012416102769800_bib13","unstructured":"ECtHR. 2020. Overview of the case-law of the ECHR. Annual Report 2020 of the European Court of Human Rights, Council of Europe."},{"issue":"4","key":"2023012416102769800_bib14","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1007\/s10506-007-9039-z","article-title":"Extractive summarisation of legal texts","volume":"14","author":"Hachey","year":"2006","journal-title":"Artificial Intelligence and Law"},{"key":"2023012416102769800_bib15","article-title":"Algorithms in the criminal justice system: Assessing the use of risk assessments in sentencing","author":"Kehl","year":"2017"},{"key":"2023012416102769800_bib16","article-title":"Adam: A method for stochastic optimization","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR)","author":"Kingma","year":"2015"},{"issue":"1","key":"2023012416102769800_bib17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2307\/1951767","article-title":"Predicting supreme court decisions mathematically: A quantitative analysis of the \u201cright to counsel\u201d cases","volume":"51","author":"Kort","year":"1957","journal-title":"American Political Science Review"},{"key":"2023012416102769800_bib18","article-title":"Precedent and analogy in legal reasoning","volume-title":"The Stanford Encyclopedia of Philosophy","author":"Lamond","year":"2016"},{"issue":"4","key":"2023012416102769800_bib19","first-page":"337","article-title":"What computers can do: Analysis and prediction of judicial decisions","volume":"49","author":"Lawlor","year":"1963","journal-title":"American Bar Association Journal"},{"key":"2023012416102769800_bib20","article-title":"The role of precedent at the European court of human rights: A network analysis of case citations","author":"Lupu","year":"2010","journal-title":"OpenSIUC"},{"key":"2023012416102769800_bib21","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1007\/978-3-642-00382-0_40","article-title":"NLP for Shallow Question Answering of Legal Documents Using Graphs","author":"Monroy","year":"2009"},{"key":"2023012416102769800_bib22","first-page":"1006","article-title":"Applying correlation analysis to case prediction","volume":"42","author":"Nagel","year":"1963","journal-title":"Texas Law Review"},{"key":"2023012416102769800_bib23","first-page":"8026","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume-title":"Advances in Neural Information Processing Systems","author":"Paszke","year":"2019"},{"key":"2023012416102769800_bib24","doi-asserted-by":"publisher","first-page":"2275","DOI":"10.18653\/v1\/2021.naacl-main.181","article-title":"What about the precedent: An information-theoretic analysis of common law","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Valvoda","year":"2021"},{"key":"2023012416102769800_bib25","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1007\/978-3-319-93794-6_2","article-title":"From case law to ratio decidendi","volume-title":"New Frontiers in Artificial Intelligence","author":"Valvoda","year":"2018"},{"key":"2023012416102769800_bib26","first-page":"141","article-title":"Using agreement statements to identify majority opinion in UKHL case law","volume-title":"Legal Knowledge and Information Systems","author":"Valvoda","year":"2018"},{"key":"2023012416102769800_bib27","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"2023012416102769800_bib28","doi-asserted-by":"publisher","first-page":"38","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"Transformers: State-of-the-art natural language processing","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Wolf","year":"2020"},{"key":"2023012416102769800_bib29","article-title":"CAIL2018: A large-scale legal dataset for judgment prediction","author":"Xiao","year":"2018","journal-title":"ArXiv"},{"key":"2023012416102769800_bib30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.280","article-title":"Distinguish confusing law articles for legal judgment prediction","author":"Nuo","year":"2020","journal-title":"arXiv preprint arXiv:2004 .02557"},{"key":"2023012416102769800_bib31","doi-asserted-by":"publisher","first-page":"3540","DOI":"10.18653\/v1\/D18-1390","article-title":"Legal judgment prediction via topological learning","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Zhong","year":"2018"},{"key":"2023012416102769800_bib32","article-title":"In the context of the common law: The European court of human rights in Strasbourg transcript","author":"Zupancic","year":"2016","journal-title":"Transcript of a lecture given at Gresham College"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00532\/2067838\/tacl_a_00532.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00532\/2067838\/tacl_a_00532.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,24]],"date-time":"2023-01-24T16:18:40Z","timestamp":1674577120000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00532\/114591\/On-the-Role-of-Negative-Precedent-in-Legal-Outcome"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":32,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00532","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}