{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T16:13:48Z","timestamp":1778256828581,"version":"3.51.4"},"reference-count":46,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T00:00:00Z","timestamp":1743465600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["UIDB\/04152\/2020"],"award-info":[{"award-number":["UIDB\/04152\/2020"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["TSI-2022-AI4IA-EU-IB-22PT09"],"award-info":[{"award-number":["TSI-2022-AI4IA-EU-IB-22PT09"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Union","doi-asserted-by":"publisher","award":["UIDB\/04152\/2020"],"award-info":[{"award-number":["UIDB\/04152\/2020"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Union","doi-asserted-by":"publisher","award":["TSI-2022-AI4IA-EU-IB-22PT09"],"award-info":[{"award-number":["TSI-2022-AI4IA-EU-IB-22PT09"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Technologies"],"abstract":"<jats:p>Legislative impact assessment (LIA) can be defined as the process performed by governments and legislative bodies to evaluate the potential effects of proposed policies or directives before they are implemented. This assessment typically covers various aspects (including economic, social, and environmental impacts) and is designed to ensure that policy proposals are well-founded, transparent, and that potential impacts are thoroughly examined before decisions are made. This process is nowadays performed by human experts and requires a significant amount of time. It is also characterized by some subjectivity that makes it difficult for citizens and companies to perceive the process as a transparent one. Moreover, public administration services responsible for LIA recognize significant difficulties in performing a timely and effective impact assessment exercise due to the lack of human and financial resources. To answer this call, this paper presents an artificial intelligence-based system to automatizing part of the impact assessment process, with the specific objective of detecting administrative burdens from transposed EU legislation. The system is built on a fine-tuned, transformer-based architecture leveraging transfer learning, making it an innovative tool for automating legislative impact assessment. Comprehensive testing on transposed European legislation demonstrated that the system significantly enhances efficiency and accuracy in what has traditionally been a complex and time-consuming task.<\/jats:p>","DOI":"10.3390\/technologies13040134","type":"journal-article","created":{"date-parts":[[2025,4,2]],"date-time":"2025-04-02T04:06:05Z","timestamp":1743566765000},"page":"134","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["A Transformer-Based Model for the Automatic Detection of Administrative Burdens in Transposed Legislative Documents"],"prefix":"10.3390","volume":"13","author":[{"given":"Victor","family":"Costa","sequence":"first","affiliation":[{"name":"NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa, Campus de Campolide, 1070-332 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8793-1451","authenticated-orcid":false,"given":"Mauro","family":"Castelli","sequence":"additional","affiliation":[{"name":"NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa, Campus de Campolide, 1070-332 Lisboa, Portugal"}]},{"given":"Pedro","family":"Coelho","sequence":"additional","affiliation":[{"name":"NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa, Campus de Campolide, 1070-332 Lisboa, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kirkpatrick, C.H., and Parker, D. (2007). Regulatory Impact Assessment: Towards Better Regulation?, Edward Elgar Publishing.","DOI":"10.4337\/9781847208774.00007"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1111\/j.1748-5991.2011.01123.x","article-title":"The many uses of regulatory impact assessment: A meta-analysis of EU and UK cases","volume":"6","author":"Dunlop","year":"2012","journal-title":"Regul. Gov."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1017\/S1867299X00004736","article-title":"How much better is better regulation? Assessing the impact of the better regulation package on the European Union\u2014A research agenda","volume":"6","author":"Alemanno","year":"2015","journal-title":"Eur. J. Risk Regul."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1017\/err.2021.40","article-title":"Can we better the European Union better regulation agenda?","volume":"12","author":"Simonelli","year":"2021","journal-title":"Eur. J. Risk Regul."},{"key":"ref_5","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.aiopen.2022.10.001","article-title":"A survey of transformers","volume":"3","author":"Lin","year":"2022","journal-title":"AI Open"},{"key":"ref_7","unstructured":"Devlin, J. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/s40537-023-00842-0","article-title":"Survey of transformers and towards ensemble learning using transformers for natural language processing","volume":"11","author":"Zhang","year":"2024","journal-title":"J. Big Data"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Krasadakis, P., Sakkopoulos, E., and Verykios, V.S. (2024). A survey on challenges and advances in natural language processing with a focus on legal informatics and low-resource languages. Electronics, 13.","DOI":"10.3390\/electronics13030648"},{"key":"ref_10","unstructured":"European Comission Action Programme for Reducing Administrative Burdens in the EU Final Report (2025, January 09). Accompanying the Document Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. EU Regulatory Fitness. COMMISSION STAFF WORKING DOCUMENT. SWD (2012). Available online: https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/HTML\/?uri=CELEX:52012SC0423."},{"key":"ref_11","unstructured":"Hradec, J., Ostlaender, N., Macmillan, C., Acs, S., Listorti, G., Tomas, R., and Arnes Novau, X. (2019). Semantic Text Analysis Tool: SeTA, Publications Office of the European Union."},{"key":"ref_12","unstructured":"Levy, O., and Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. Adv. Neural Inf. Process. Syst., 27."},{"key":"ref_13","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv."},{"key":"ref_14","unstructured":"Ostlaender, N., Acs, S., Listorti, G., Hardy, M., Ghirimoldi, G., Hradec, J., and Smits, P. (2019). Modelling Inventory and Knowledge Management System of the European Commission (MIDAS), Publications Office of the European Union."},{"key":"ref_15","unstructured":"Acs, S., Ostlaender, N., Listorti, G., Hradec, J., Hardy, M., Smits, P., and Hordijk, L. (2019). Modelling for EU Policy Support: Impact Assessments, Publications Office of the European Union. Technical Report."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Misuraca, G., and Van Noordt, C. (2020). AI Watch-Artificial Intelligence in Public Services: Overview of the Use and Impact of AI in Public Services in the EU, Publications Office of the European Union. JRC Working Papers.","DOI":"10.1145\/3428502.3428513"},{"key":"ref_17","unstructured":"Romic, D., and Halak, Z.V. (2014). Regulatory Impact Assessment in the Republic of Croatia-Situation and Perspective. Chall. Knowl. Soc., 880\u2013891."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zeitz, D. (2016). Better Regulation in Germany as Quality Assurance System: Recent Development and Current Challenges, Netherlands Administrative Law Library (Nall).","DOI":"10.5553\/NALL\/.000025"},{"key":"ref_19","first-page":"14","article-title":"Regulatory impact assessment: Current situation and prospects in the German Parliament","volume":"101","author":"Karpen","year":"2015","journal-title":"Amic. Curiae"},{"key":"ref_20","first-page":"1","article-title":"Regulatory policy in the context of effective public governance: Evidence of Eastern European Countries","volume":"40","author":"Kosach","year":"2022","journal-title":"Cuest. Pol\u00edticas"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1109\/TNNLS.2020.2979670","article-title":"A survey of the usages of deep learning for natural language processing","volume":"32","author":"Otter","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.eng.2022.04.024","article-title":"Pre-trained language models and their applications","volume":"25","author":"Wang","year":"2023","journal-title":"Engineering"},{"key":"ref_23","unstructured":"Lipton, Z.C., Berkowitz, J., and Elkan, C. (2015). A critical review of recurrent neural networks for sequence learning. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"6999","DOI":"10.1109\/TNNLS.2021.3084827","article-title":"A survey of convolutional neural networks: Analysis, applications, and prospects","volume":"33","author":"Li","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.neucom.2021.03.091","article-title":"A review on the attention mechanism of deep learning","volume":"452","author":"Niu","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rodrigues, J., Gomes, L., Silva, J., Branco, A., Santos, R., Cardoso, H.L., and Os\u00f3rio, T. (2023, January 5\u20138). Advancing neural encoding of portuguese with transformer albertina pt. Proceedings of the EPIA Conference on Artificial Intelligence, Faial Island, Portugal.","DOI":"10.1007\/978-3-031-49008-8_35"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Souza, F., Nogueira, R., and Lotufo, R. (2020, January 20\u201323). BERTimbau: Pretrained BERT models for Brazilian Portuguese. Proceedings of the Intelligent Systems: 9th Brazilian Conference, BRACIS 2020, Rio Grande, Brazil. Proceedings, Part I 9.","DOI":"10.1007\/978-3-030-61377-8_28"},{"key":"ref_28","unstructured":"Lan, Z. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_30","unstructured":"Alyafeai, Z., AlShaibani, M.S., and Ahmad, I. (2020). A survey on transfer learning in natural language processing. arXiv."},{"key":"ref_31","unstructured":"Koroteev, M.V. (2021). BERT: A review of applications in natural language processing and understanding. arXiv."},{"key":"ref_32","unstructured":"Santos, R., Rodrigues, J., Gomes, L., Silva, J., Branco, A., Cardoso, H.L., Os\u00f3rio, T.F., and Leite, B. (2024). Fostering the Ecosystem of Open Neural Encoders for Portuguese with Albertina PT* Family. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"100048","DOI":"10.1016\/j.nlp.2023.100048","article-title":"A survey of GPT-3 family large language models including ChatGPT and GPT-4","volume":"6","author":"Kalyan","year":"2024","journal-title":"Nat. Lang. Process. J."},{"key":"ref_34","unstructured":"Shaheen, Z., Wohlgenannt, G., and Filtz, E. (2020). Large scale legal text classification using transformer models. arXiv."},{"key":"ref_35","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Bommarito II, M.J., Katz, D.M., and Detterman, E.M. (2021). LexNLP: Natural language processing and information extraction for legal and regulatory texts. Research Handbook on Big Data Law, Edward Elgar Publishing.","DOI":"10.4337\/9781788972826.00017"},{"key":"ref_37","unstructured":"Zheng, L., Guha, N., Anderson, B.R., Henderson, P., and Ho, D.E. (2021, January 21\u201325). Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset of 53,000+ Legal Holdings. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law, S\u00e3o Paulo, Brazil."},{"key":"ref_38","first-page":"29217","article-title":"Pile of law: Learning responsible data filtering from the law and a 256 gb open-source legal dataset","volume":"35","author":"Henderson","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_39","unstructured":"Ratner, A., Hancock, B., Dunnmon, J., Sala, F., Pandey, S., and R\u00e9, C. (February, January 27). Training complex models with multi-task weak supervision. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_40","unstructured":"Settles, B. (2011, January 16). From theories to queries: Active learning in practice. Proceedings of the Active Learning and Experimental Design Workshop in Conjunction with AISTATS 2010, Sardinia, Italy. JMLR Workshop and Conference Proceedings."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1373","DOI":"10.1613\/jair.1.12125","article-title":"Confident learning: Estimating uncertainty in dataset labels","volume":"70","author":"Northcutt","year":"2021","journal-title":"J. Artif. Intell. Res."},{"key":"ref_42","first-page":"1","article-title":"A survey of deep active learning","volume":"54","author":"Ren","year":"2021","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Shakeel, F., Sabhitha, A.S., and Sharma, S. (2017, January 3\u20135). Exploratory review on class imbalance problem: An overview. Proceedings of the 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Delhi, India.","DOI":"10.1109\/ICCCNT.2017.8204150"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Feurer, M., and Hutter, F. (2019). Hyperparameter Optimization, Springer International Publishing. Automated Machine Learning Methods Systems Challenges.","DOI":"10.1007\/978-3-030-05318-5_1"},{"key":"ref_45","unstructured":"Loshchilov, I., and Hutter, F. (2017). Fixing weight decay regularization in adam. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3052775","article-title":"Idf for word n-grams","volume":"36","author":"Shirakawa","year":"2017","journal-title":"ACM Trans. Inf. Syst. (TOIS)"}],"container-title":["Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-7080\/13\/4\/134\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:07:57Z","timestamp":1760029677000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-7080\/13\/4\/134"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,1]]},"references-count":46,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["technologies13040134"],"URL":"https:\/\/doi.org\/10.3390\/technologies13040134","relation":{},"ISSN":["2227-7080"],"issn-type":[{"value":"2227-7080","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,1]]}}}