{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T09:08:11Z","timestamp":1774429691020,"version":"3.50.1"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T00:00:00Z","timestamp":1731369600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T00:00:00Z","timestamp":1731369600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["101006828-Flying Forward 2020"],"award-info":[{"award-number":["101006828-Flying Forward 2020"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Law"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The topic of annotating legal data has received surprisingly little attention. A key challenge of the annotation process is reaching a sufficient agreement between annotators and filtering mistakes from genuine disagreement. This study presents an approach that provides insights into and resolves potential disagreement amongst annotators. It (1) introduces different strategies to calculate agreement levels and compares (2) agreement levels between annotators (inter-annotator agreement) before and after a revision round and (3) agreement levels for annotators who annotate the same texts twice (intra-annotator agreement). The inter-annotator agreement levels are compared to a revision round in which an arbiter corrected the annotator\u2019s labels. The analysis is based on the annotation of EU legislative provisions at two stages (initial annotations, after annotator revisions) and for various tasks (Definitions, References, Quantities, IF-THEN statements, Exceptions, Scope, Hierarchy, Deontic Clauses, Active and Passive Role) by multiple annotators. The results reveal that agreement levels vary based on the stage of measurement (before\/after revisions), the nature of the task, the method of assessment, and the annotator combination. The agreement scores - along with some initial measurements\u2014align with those reported in previous research but increase after each revision round. This suggests that annotator revisions can substantially reduce disagreement. Additionally, disagreements were found not only between but also among annotators. This inconsistency does not appear to stem from a lack of understanding of the guidelines or a lack of seriousness in task execution, as evidenced by moderate to substantial inter-annotator agreement scores. These findings suggest that annotators identified multiple valid interpretations, which highlights the complexity of annotating legislative provisions. The results underscore the significance of embracing, addressing, and reporting about (dis)agreement in different ways and at the various stages of an annotation task.<\/jats:p>","DOI":"10.1007\/s10506-024-09423-9","type":"journal-article","created":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T01:09:58Z","timestamp":1731373798000},"page":"191-226","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Deciphering disagreement in the annotation of EU legislation"],"prefix":"10.1007","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4102-4415","authenticated-orcid":false,"given":"Gijs","family":"van Dijck","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carlos","family":"Aguilera","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shashank M.","family":"Chakravarthy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,11,12]]},"reference":[{"issue":"4","key":"9423_CR1","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1007\/s10506-017-9205-x","volume":"25","author":"JP Aires","year":"2017","unstructured":"Aires JP, Pinheiro D, Lima VSD, Meneguzzi F (2017) Norm conflict identification in contracts. Artif Intell Law 25(4):397\u2013428. https:\/\/doi.org\/10.1007\/s10506-017-9205-x","journal-title":"Artif Intell Law"},{"key":"9423_CR2","doi-asserted-by":"crossref","unstructured":"Akhtar S , Basile V, Patti V (2020) Modeling annotator perspective and polarized opinions to improve hate speech detection [Conference Proceedings]. In: Proceedings of the aaai conference on human computation and crowdsourcing (Vol.\u00a08, p.151\u2013154). https:\/\/ojs.aaai.org\/index.php\/HCOMP\/article\/view\/7473","DOI":"10.1609\/hcomp.v8i1.7473"},{"key":"9423_CR3","unstructured":"Amantea IA , Di\u00a0Caro L , Humphreys L , Nanda R, Sulis E (2019) Modelling norm types and their inter-relationships in eu directives [Conference Proceedings]. Asail@icail. https:\/\/ceur-ws.org\/Vol-2385\/paper8.pdf"},{"key":"9423_CR4","doi-asserted-by":"crossref","unstructured":"Athan T, Boley H, Governatori G, Palmirani M, Paschke A, Wyner A (2013) Legalruleml: from metamodel to use cases [Conference Proceedings]. L\u00a0Morgenstern, P\u00a0Stefaneas, F\u00a0L\u00e9vy, A\u00a0Wyner, & A\u00a0Paschke (Eds.), Theory, practice, and applications of rules on the web (p.13-18). Springer Berlin Heidelberg. https:\/\/link.springer.com\/chapter\/10.1007\/978-3-642-39617-5_4","DOI":"10.1007\/978-3-642-39617-5_4"},{"key":"9423_CR5","unstructured":"Bakker R, van Drie R.A, de Boer M, van Doesburg R, van Engers T (2022) Semantic role labelling for dutch law texts [Conference Proceedings]. (p.448-457). European Language Resources Association. https:\/\/aclanthology.org\/2022.lrec-1.47"},{"issue":"1145\/1165485","key":"9423_CR6","first-page":"1165506","volume":"10","author":"C Biagioli","year":"2005","unstructured":"Biagioli C, Francesconi E, Passerini A, Montemagni S, Soria C (2005) Automatic semantics extraction in law documents. Assoc Comput Mach 10(1145\/1165485):1165506","journal-title":"Assoc Comput Mach"},{"key":"9423_CR7","doi-asserted-by":"publisher","unstructured":"Boella G, Humphreys L, Martin M, Rossi P, van\u00a0der Torre L (2012) Eunomos, a legal document and knowledge management system to build legal services [Book Section]. In M\u00a0Palmirani, U\u00a0Pagallo, P\u00a0Casanovas, & G\u00a0Sartor (Eds.), Ai approaches to the complexity of legal systems. models and ethical challenges for legal systems, legal language and legal ontologies, argumentation and software agents (p.131\u2013146). Berlin, HeidelbergSpringer Berlin Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-35731-2_9","DOI":"10.1007\/978-3-642-35731-2_9"},{"key":"9423_CR8","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-023-09369-4","author":"D Braun","year":"2023","unstructured":"Braun D (2023) I beg to differ: how disagreement is handled in the annotation of legal machine learning data sets. Artif Intell Law. https:\/\/doi.org\/10.1007\/s10506-023-09369-4","journal-title":"Artif Intell Law"},{"key":"9423_CR9","doi-asserted-by":"publisher","unstructured":"Chalkidis I, Androutsopoulos I, Michos A (2018) Obligation and prohibition extraction using hierarchical rnns [Conference Proceedings]. (p.254\u2013259). Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/P18-2041","DOI":"10.18653\/v1\/P18-2041"},{"issue":"1","key":"9423_CR10","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Measur 20(1):37\u201346. https:\/\/doi.org\/10.1177\/001316446002000104","journal-title":"Educ Psychol Measur"},{"issue":"4","key":"9423_CR11","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1037\/h0026256","volume":"70","author":"J Cohen","year":"1968","unstructured":"Cohen J (1968) Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 70(4):213\u2013220. https:\/\/doi.org\/10.1037\/h0026256","journal-title":"Psychol Bull"},{"key":"9423_CR12","doi-asserted-by":"crossref","unstructured":"de\u00a0Araujo D.A, Rigo S.J, Muller C, Chishman R (2013) Automatic information extraction from texts with inference and linguistic knowledge acquisition rules [Conference Proceedings]. 2013 ieee\/wic\/acm international joint conferences on web intelligence (wi) and intelligent agent technologies (iat) (Vol.\u00a03, p.151-154)","DOI":"10.1109\/WI-IAT.2013.171"},{"key":"9423_CR13","doi-asserted-by":"crossref","unstructured":"de Maat E, Krabben K, Winkels R (2010). Machine learning versus knowledge based classification of legal texts. In: Proceedings of the 2010 conference on legal knowledge and information systems: Jurix 2010: The twenty-third annual conference (p.87-96). IOS Press","DOI":"10.3233\/978-1-60750-682-9-87"},{"key":"9423_CR14","doi-asserted-by":"crossref","unstructured":"Dragoni M, Villata S, Rizzi W, Governatori G (2018). Combining natural language processing approaches for rule extraction from legal documents [Conference Proceedings]. U\u00a0Pagallo, M\u00a0Palmirani, P\u00a0Casanovas, G\u00a0Sartor, & S\u00a0Villata (Eds.), Ai approaches to the complexity of legal systems (p.287-300). Springer International Publishing","DOI":"10.1007\/978-3-030-00178-0_19"},{"issue":"5","key":"9423_CR15","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","volume":"76","author":"JL Fleiss","year":"1971","unstructured":"Fleiss JL (1971) Measuring nominal scale agreement among many raters [Journal Article]. Psychol Bull 76(5):378\u2013382. https:\/\/doi.org\/10.1037\/h0031619","journal-title":"Psychol Bull"},{"key":"9423_CR16","unstructured":"Francesconi E (2010). Legal rules learning based on a semantic model for legislation. In: Proceedings of the lrec 2010 workshop on the semantic processing of legal texts (splet-2010). http:\/\/lrec.elra.info\/proceedings\/lrec2010\/workshops\/W23.pdf#page=51"},{"issue":"1","key":"9423_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10506-007-9038-0","volume":"15","author":"E Francesconi","year":"2007","unstructured":"Francesconi E, Passerini A (2007) Automatic classification of provisions in legislative texts [Journal Article]. Artif Intell Law 15(1):1\u201317. https:\/\/doi.org\/10.1007\/s10506-007-9038-0","journal-title":"Artif Intell Law"},{"key":"9423_CR18","doi-asserted-by":"crossref","unstructured":"Gao X, Singh MP (2014) Extracting normative relationships from business contracts. In: International foundation for autonomous agents and multiagent systems","DOI":"10.65109\/ZNOK3576"},{"key":"9423_CR19","doi-asserted-by":"publisher","unstructured":"Ghanavati S (2013) Legal-urn framework for legal compliance of business processes Thesis. https:\/\/doi.org\/10.1007\/978-3-642-35731-2_9","DOI":"10.1007\/978-3-642-35731-2_9"},{"key":"9423_CR20","doi-asserted-by":"publisher","DOI":"10.1145\/2593770.2593780","author":"S Ghanavati","year":"2014","unstructured":"Ghanavati S, Amyot D, Rifaut A (2014) Legal goal-oriented requirement language (legal grl) for modeling regulations. Assoc Comput Mach. https:\/\/doi.org\/10.1145\/2593770.2593780","journal-title":"Assoc Comput Mach"},{"key":"9423_CR21","doi-asserted-by":"crossref","unstructured":"Glaser I, Scepankova E, Matthes F (2018) Classifying semantic types of legal sentences: portability of machine learning models [Conference Proceedings]. Jurix (p.61\u201370). https:\/\/ebooks.iospress.nl\/publication\/50835","DOI":"10.3233\/978-1-61499-935-5-61"},{"key":"9423_CR22","unstructured":"Grabmair M, Ashley K, Hwa R, Sweeney PM (2011) Toward extracting information from public health statutes using text classification machine learning. Legal knowledge and information systems-jurix 2011: the twenty-fourth annual conference, university of vienna, austria, 14th-16th dec 2011 (Vol.\u00a0235, p.73-82)"},{"key":"9423_CR23","unstructured":"Humphreys L (2016) Populating legal ontologies using information extraction based on semantic role labeling and text similarity Thesis. https:\/\/orbilu.uni.lu\/bitstream\/10993\/33810\/1\/LlBH-thesis-2016.pdf"},{"issue":"2","key":"9423_CR24","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/s10506-020-09271-3","volume":"29","author":"L Humphreys","year":"2021","unstructured":"Humphreys L, Boella G, van der Torre L, Robaldo L, Di Caro L, Ghanavati S, Muthuri R (2021) Populating legal ontologies using semantic role labeling. Artif Intell Law 29(2):171\u2013211. https:\/\/doi.org\/10.1007\/s10506-020-09271-3","journal-title":"Artif Intell Law"},{"key":"9423_CR25","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473921","author":"V Joshi","year":"2021","unstructured":"Joshi V, Anish PR, Ghaisas S (2021) Domain adaptation for an automated classification of deontic modalities in software engineering contracts. Assoc Comput Mach. https:\/\/doi.org\/10.1145\/3468264.3473921","journal-title":"Assoc Comput Mach"},{"key":"9423_CR26","doi-asserted-by":"crossref","unstructured":"Kiyavitskaya N, Zeni N, Breaux T.D, Ant\u00f3n A.I, Cordy J.R, Mich L, Mylopoulos J (2008) Automating the extraction of rights and obligations for regulatory compliance [Conference Proceedings]. Q\u00a0Li, S\u00a0Spaccapietra, E\u00a0Yu, & A\u00a0Oliv\u00e9 (Eds.), Conceptual modeling - er 2008 (p.154\u2013168). Springer Berlin Heidelberg","DOI":"10.1007\/978-3-540-87877-3_13"},{"key":"9423_CR27","doi-asserted-by":"publisher","unstructured":"Krippendorff K (2019) Content analysis: An introduction to its methodology (Fourth Edition ed.) [Book]. Thousand Oaks, California. https:\/\/doi.org\/10.4135\/9781071878781","DOI":"10.4135\/9781071878781"},{"issue":"1","key":"9423_CR28","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159\u2013174. https:\/\/doi.org\/10.2307\/2529310","journal-title":"Biometrics"},{"key":"9423_CR29","doi-asserted-by":"publisher","unstructured":"Liga D, Palmirani M (2022) Transfer learning for deontic rule classification: The case study of the gdpr. In: E\u00a0Francesconi, G\u00a0Borges, & C\u00a0Sorge (Eds.), Legal knowledge and information systems\u2014jurix 2022: The thirty-fifth annual conference, saarbr\u00fccken, germany, 14-16 december 2022 (Vol.\u00a0362, p.200-205). IOS Press. https:\/\/doi.org\/10.3233\/FAIA220467","DOI":"10.3233\/FAIA220467"},{"key":"9423_CR30","doi-asserted-by":"crossref","unstructured":"Liga D, Palmirani M (2023). Deontic sentence classification using tree kernel classifiers [Conference Proceedings]. K\u00a0Arai (Ed.), Intelligent systems and applications (p.54\u201373). Springer International Publishing","DOI":"10.1007\/978-3-031-16072-1_4"},{"key":"9423_CR31","doi-asserted-by":"crossref","unstructured":"Minkova K, Chakravarthy S, van Dijck G (2023) Low-resource deontic modality classification in EU legislation. D\u00a0Preo\u0163iuc-Pietro, C\u00a0Goanta, I\u00a0Chalkidis, L\u00a0Barrett, GJ\u00a0Spanakis, & N\u00a0Aletras (Eds.), Proceedings of the natural legal language processing workshop 2023 (pp. 149\u2013158). SingaporeAssociation for Computational Linguistics. https:\/\/aclanthology.org\/2023.nllp-1.15","DOI":"10.18653\/v1\/2023.nllp-1.15"},{"key":"9423_CR32","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2022.105771","author":"A Mowbray","year":"2023","unstructured":"Mowbray A, Chung P, Greenleaf G (2023) Explainable ai (xai) in rules as code (rac): The datalex approach. Comput Law Secur Rev. https:\/\/doi.org\/10.1016\/j.clsr.2022.105771","journal-title":"Comput Law Secur Rev"},{"key":"9423_CR33","doi-asserted-by":"publisher","unstructured":"Mowbray A, Greenleaf G, Chung P (2021) Law as code: Introducing austlii\u2019s datalex ai [Unpublished Work]. https:\/\/doi.org\/10.2139\/ssrn.3971919","DOI":"10.2139\/ssrn.3971919"},{"key":"9423_CR34","doi-asserted-by":"publisher","unstructured":"O\u2019Neill J, Buitelaar P, Robin C, O\u2019 Brien, L (2017) Classifying sentential modality in legal language: a use case in financial regulations, acts and directives. In: Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3086512.3086528","DOI":"10.1145\/3086512.3086528"},{"key":"9423_CR35","doi-asserted-by":"publisher","unstructured":"Palmirani M, Governatori G, Rotolo A, Tabet S, Boley H, Paschke A (2011) Legalruleml: Xml-based rules and norms [Book Section]. In F\u00a0Olken, M\u00a0Palmirani, & D\u00a0Sottara (Eds.), Rule-based modeling and computing on the semantic web (p.298-312). Berlin, HeidelbergSpringer Berlin Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-24908-2_30","DOI":"10.1007\/978-3-642-24908-2_30"},{"key":"9423_CR36","unstructured":"Peters W, Wyner A. (2016). Legal text interpretation: Identifying hohfeldian relations from text [Conference Proceedings]. (p.379\u2013384). European Language Resources Association (ELRA). https:\/\/aclanthology.org\/L16-1059"},{"key":"9423_CR37","doi-asserted-by":"crossref","unstructured":"Ramponi A, Leonardelli E (2022) Dh-fbk at semeval-2022 task 4: Leveraging annotators\u2019 disagreement and multiple data views for patronizing language detection [Conference Proceedings]. (p.324-334). Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/2022.semeval-1.42","DOI":"10.18653\/v1\/2022.semeval-1.42"},{"key":"9423_CR38","doi-asserted-by":"crossref","unstructured":"Shaghaghian S, Feng L, Jafarpour B, Pogrebnyakov N (2020) Customizing contextualized language models for legal document reviews [Conference Proceedings]. 2020 ieee international conference on big data (big data) (p.2139-2148)","DOI":"10.1109\/BigData50022.2020.9378201"},{"key":"9423_CR39","doi-asserted-by":"crossref","unstructured":"Sleimi A, Sannier N, Sabetzadeh M, Briand L, Dann J (2018) Automated extraction of semantic legal metadata using natural language processing. In: 2018 ieee 26th international requirements engineering conference (re) (p.124\u2013135)","DOI":"10.1109\/RE.2018.00022"},{"issue":"4","key":"9423_CR40","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2023.103374","volume":"60","author":"J Sun","year":"2023","unstructured":"Sun J, Huang S, Wei C (2023) A bert-based deontic logic learner. Inf Process Manag 60(4):103374. https:\/\/doi.org\/10.1016\/j.ipm.2023.103374","journal-title":"Inf Process Manag"},{"key":"9423_CR41","doi-asserted-by":"crossref","unstructured":"Tiscornia D, Turchi F (1997) Formalization of legislative documents based on a functional model. In: Proceedings of the 6th international conference on artificial intelligence and law (p.63-71). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/261618.261633","DOI":"10.1145\/261618.261633"},{"key":"9423_CR42","unstructured":"van Doesburg R, van Engers TM (2019) Explicit interpretation of the dutch aliens act [Conference Proceedings]. Proceedings of the workshop on artificial intelligence and the administrative state co-located with 17th international conference on ai and law (aias@icail). https:\/\/ceur-ws.org\/Vol-2471\/paper6.pdf"},{"issue":"1","key":"9423_CR43","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1007\/s10506-018-9228-y","volume":"27","author":"B Waltl","year":"2019","unstructured":"Waltl B, Bonczek G, Scepankova E, Matthes F (2019) Semantic types of legal norms in german laws: classification and analysis using local linear explanations [Journal Article]. Artif Intell Law 27(1):43\u201371. https:\/\/doi.org\/10.1007\/s10506-018-9228-y","journal-title":"Artif Intell Law"},{"key":"9423_CR44","unstructured":"Waltl B, Landthaler J, Matthes F (2016) Differentiation and empirical analysis of reference types in legal documents. In: International conference on legal knowledge and information systems"},{"key":"9423_CR45","unstructured":"Waltl B, Muhr J, Glaser I, Bonczek G, Scepankova E, Matthes F (2017) Classifying legal norms with active machine learning [Book]"},{"key":"9423_CR46","unstructured":"Wyner A, Peters W (2011) On rule extraction from regulations [Conference Proceedings]. Frontiers in artificial intelligence and applications (Vol. Volume 235: Legal Knowledge and Information Systems, p.113\u2013122)"}],"container-title":["Artificial Intelligence and Law"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10506-024-09423-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10506-024-09423-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10506-024-09423-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T06:44:53Z","timestamp":1774421093000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10506-024-09423-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,12]]},"references-count":46,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,3]]}},"alternative-id":["9423"],"URL":"https:\/\/doi.org\/10.1007\/s10506-024-09423-9","relation":{},"ISSN":["0924-8463","1572-8382"],"issn-type":[{"value":"0924-8463","type":"print"},{"value":"1572-8382","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,12]]},"assertion":[{"value":"6 October 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 November 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no Conflict of interest. The funding organization had no role in the design of the study, in the collection, analysis, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}