{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T18:37:56Z","timestamp":1794076676835,"version":"4.8.0"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T00:00:00Z","timestamp":1599177600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["06487-2017"],"award-info":[{"award-number":["06487-2017"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008240","name":"Fonds de Recherche du Qu\u00e9bec-Soci\u00e9t\u00e9 et Culture","doi-asserted-by":"publisher","award":["Audace 2018"],"award-info":[{"award-number":["Audace 2018"]}],"id":[{"id":"10.13039\/100008240","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Stats"],"abstract":"<jats:p>On average, one in three Canadians will be affected by a legal problem over a three-year period. Unfortunately, whether it is legal representation or legal advice, the very high cost of these services excludes disadvantaged and most vulnerable people, forcing them to represent themselves. For these people, accessing legal information is therefore critical. In this work, we attempt to tackle this problem by embedding legal data in a conversational interface. We introduce two dialog systems (chatbots) created to provide legal information. The first one, based on data from the Government of Canada, deals with immigration issues, while the second one informs bank employees about legal issues related to their job tasks. Both chatbots rely on various representations and classification algorithms, from mature techniques to novel advances in the field. The chatbot dedicated to immigration issues is shared with the research community as an open resource project.<\/jats:p>","DOI":"10.3390\/stats3030023","type":"journal-article","created":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T11:24:24Z","timestamp":1599218664000},"page":"356-375","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Improving Access to Justice with Legal Chatbots"],"prefix":"10.3390","volume":"3","author":[{"given":"Marc","family":"Queudot","sequence":"first","affiliation":[{"name":"PK-4935, 201 Avenue du President-Kennedy, Universit\u00e9 du Qu\u00e9bec \u00e0 Montr\u00e9al UQAM, Montreal, QC H2X 3Y7, Canada"},{"name":"600 Rue De la Gaucheti\u00e8re Ouest, National Bank of Canada, Montr\u00e9al, QC H3B 4L2, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"\u00c9ric","family":"Charton","sequence":"additional","affiliation":[{"name":"600 Rue De la Gaucheti\u00e8re Ouest, National Bank of Canada, Montr\u00e9al, QC H3B 4L2, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8196-2153","authenticated-orcid":false,"given":"Marie-Jean","family":"Meurs","sequence":"additional","affiliation":[{"name":"PK-4935, 201 Avenue du President-Kennedy, Universit\u00e9 du Qu\u00e9bec \u00e0 Montr\u00e9al UQAM, Montreal, QC H2X 3Y7, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"495","DOI":"10.7202\/1052476ar","article-title":"Agir seul en justice: Du droit au choix\u2014\u00c9tat de la jurisprudence sur les droits des justiciables non repr\u00e9sent\u00e9s","volume":"59","author":"Laniel","year":"2018","journal-title":"Les Cah. Droit"},{"key":"ref_2","first-page":"585","article-title":"Illiberal Construction of Pro Se Pleadings","volume":"159","author":"Schneider","year":"2010","journal-title":"Univ. Pa. Law Rev."},{"key":"ref_3","unstructured":"Manning, C.D., Manning, C.D., and Sch\u00fctze, H. (1999). Foundations of Statistical Natural Language Processing, MIT Press."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Manning, C.D., Raghavan, P., and Sch\u00fctze, H. (2008). Introduction to Information Retrieval, Cambridge University Press.","DOI":"10.1017\/CBO9780511809071"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1108\/eb026526","article-title":"A Statistical Interpretation of Term Specificity and its Application in Retrieval","volume":"28","author":"Jones","year":"1972","journal-title":"J. Doc."},{"key":"ref_6","unstructured":"Shannon, C.E., and Weaver, W. (1998). A Mathematical Theory of Communication, University of Illinois Press."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1108\/00220410410560582","article-title":"Understanding inverse document frequency: On theoretical arguments for IDF","volume":"60","author":"Robertson","year":"2004","journal-title":"J. Doc."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1080\/00437956.1954.11659520","article-title":"Distributional Structure","volume":"10","author":"Harris","year":"1954","journal-title":"Word"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1037\/0033-295X.104.2.211","article-title":"A Solution to Plato\u2019s Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge","volume":"104","author":"Landauer","year":"1997","journal-title":"Psychol. Rev."},{"key":"ref_10","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv."},{"key":"ref_11","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5\u201310). Distributed Representations of Words and Phrases and Their Compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_12","unstructured":"Arora, S., Liang, Y., and Ma, T. (2017, January 24\u201326). A Simple but Tough-to-Beat Baseline for Sentence Embeddings. Proceedings of the ICLR 2017 Conference Submission, Toulon, France."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wu, L.Y., Fisch, A., Chopra, S., Adams, K., Bordes, A., and Weston, J. (2018, January 2\u20137). Starspace: Embed all the things!. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11996"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kenter, T., Borisov, A., and De Rijke, M. (2016). Siamese CBOW: Optimizing word embeddings for sentence representations. arXiv.","DOI":"10.18653\/v1\/P16-1089"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Xu, W., and Rudnicky, A. (2000, January 16\u201320). Can Artificial Neural Networks Learn Language Models?. Proceedings of the Sixth International Conference on Spoken Language Processing, Beijing, China.","DOI":"10.21437\/ICSLP.2000-50"},{"key":"ref_16","first-page":"1137","article-title":"A Neural Probabilistic Language Model","volume":"3","author":"Bengio","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_18","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv."},{"key":"ref_19","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_20","unstructured":"Jurafsky, D., and Martin, J.H. (2014). Question Answering. Speech and Language Processing, Pearson."},{"key":"ref_21","unstructured":"Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., and Ives, Z. (2007, January 11\u201315). DBpedia: A Nucleus for a Web of Open Data. Proceedings of the 6th International The Semantic Web and 2nd Asian Conference on Asian Semantic Web Conference, ISWC\u201907\/ASWC\u201907, Busan, Korea."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Deriu, J., Rodrigo, A., Otegi, A., Echegoyen, G., Rosset, S., Agirre, E., and Cieliebak, M. (2019). Survey on Evaluation Methods for Dialogue Systems. arXiv.","DOI":"10.1007\/s10462-020-09866-x"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/365153.365168","article-title":"ELIZA\u2014A Computer Program for the Study of Natural Language Communication between Man and Machine","volume":"9","author":"Weizenbaum","year":"1966","journal-title":"Commun. ACM"},{"key":"ref_24","unstructured":"Wallace, R., Tomabechi, H., and Aimless, D. (2019, December 01). Chatterbots Go Native: Considerations for an Eco-System Fostering the Development of Artificial Life Forms in a Human World. Available online: http:\/\/www.pandorabots.com\/pandora\/pics\/chatterbotsgonative.doc."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Satu, M.S., and Parvez, M.H. (2015, January 26\u201327). Review of Integrated Applications With AIML Based Chatbot. Proceedings of the 2015 International Conference on Computer and Information Engineering (ICCIE), Rajshahi, Bangladesh.","DOI":"10.1109\/CCIE.2015.7399324"},{"key":"ref_26","unstructured":"Vinyals, O., and Le, Q. (2015). A Neural Conversational Model. arXiv."},{"key":"ref_27","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8\u201313). Sequence to Sequence Learning with Neural Networks. Proceedings of the Advances in Neural Information Processing Systems 27 (NIPS 2014), Montreal, QC, Canada."},{"key":"ref_28","unstructured":"Qiu, M., Li, F.L., Wang, S., Gao, X., Chen, Y., Zhao, W., Chen, H., Huang, J., and Chu, W. (August, January 30). AliMe Chat: A Sequence to Sequence and Rerank Based Chatbot Engine. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1093\/mind\/LIX.236.433","article-title":"Computing Machinery and Intelligence","volume":"59","author":"Turing","year":"1950","journal-title":"Mind"},{"key":"ref_30","unstructured":"Radziwill, N.M., and Benton, M.C. (2017). Evaluating Quality of Chatbots and Intelligent Conversational Agents. arXiv."},{"key":"ref_31","unstructured":"Ramos, R. (2017, March 13). Screw the Turing Test-Chatbots Don\u2019t Need to Act Human. VentureBeat, Available online: https:\/\/venturebeat.com\/2017\/02\/03\/screw-the-turing-test-chatbots-dont-need-to-act-human\/."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7\u201312). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_33","unstructured":"Lin, C.Y. (2004). ROUGE: A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out, Association for Computational Linguistics."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Lowe, R., Noseworthy, M., Serban, I.V., Angelard-Gontier, N., Bengio, Y., and Pineau, J. (2017). Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses. arXiv.","DOI":"10.18653\/v1\/P17-1103"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/0004-3702(77)90018-2","article-title":"GUS, a Frame-Driven Dialog System","volume":"8","author":"Bobrow","year":"1977","journal-title":"Artif. Intell."},{"key":"ref_36","first-page":"535","article-title":"Markov Logic Networks for Spoken Language Interpretation","volume":"1978","author":"Meurs","year":"2008","journal-title":"Inf. Syst. J."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Meurs, M.J., Lefevre, F., and De Mori, R. (June, January 31). Learning Bayesian Networks for Semantic Frame Composition in a Spoken Dialog System. Proceedings of the Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, CO, USA.","DOI":"10.3115\/1620853.1620872"},{"key":"ref_38","first-page":"6","article-title":"Dialog Systems and Chatbots","volume":"Volume 3","author":"Jurafsky","year":"2017","journal-title":"Speech and Language Processing"},{"key":"ref_39","unstructured":"Bocklisch, T., Faulkner, J., Pawlowski, N., and Nichol, A. (2017). Rasa: Open source language understanding and dialogue management. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1038\/533452a","article-title":"1500 Scientists Lift the Lid on Reproducibility","volume":"533","author":"Baker","year":"2016","journal-title":"Nat. News"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Gundersen, O.E., and Kjensmo, S. (2018, January 2\u20137). State of the art: Reproducibility in artificial intelligence. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11503"},{"key":"ref_42","first-page":"39","article-title":"Accelerating the Machine Learning Lifecycle with MLflow","volume":"41","author":"Zaharia","year":"2018","journal-title":"IEEE Data Eng. Bull."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Charton, E., Meurs, M.J., Jean-Louis, L., and Gagnon, M. (2014, January 23\u201325). Mutual Disambiguation for Entity Linking. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, MD, USA.","DOI":"10.3115\/v1\/P14-2078"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Queudot, M., and Meurs, M.J. (2018, January 25\u201328). Artificial Intelligence and Predictive Justice: Limitations and Perspectives. Proceedings of the International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Montreal, QC, Canada.","DOI":"10.1007\/978-3-319-92058-0_85"}],"container-title":["Stats"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2571-905X\/3\/3\/23\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:06:54Z","timestamp":1760177214000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2571-905X\/3\/3\/23"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,4]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["stats3030023"],"URL":"https:\/\/doi.org\/10.3390\/stats3030023","relation":{},"ISSN":["2571-905X"],"issn-type":[{"value":"2571-905X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,4]]}}}