{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,5,8]],"date-time":"2024-05-08T23:35:38Z","timestamp":1715211338518},"reference-count":12,"publisher":"Georg Thieme Verlag KG","issue":"03","funder":[{"name":"Robert Wood Johnson Foundation Future of Nursing Scholar's Program"},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"crossref","award":["5T15LM012502-04"],"award-info":[{"award-number":["5T15LM012502-04"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000051","name":"National Human Genome Research Institute","doi-asserted-by":"crossref","award":["5U01HG009454-03"],"award-info":[{"award-number":["5U01HG009454-03"]}],"id":[{"id":"10.13039\/100000051","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Rackham Graduate Student Research Grant"},{"name":"University of Michigan Institute for Data Science"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2021,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Background\u2003The lack of machine-interpretable representations of consent permissions precludes development of tools that act upon permissions across information ecosystems, at scale.<\/jats:p><jats:p>Objectives\u2003To report the process, results, and lessons learned while annotating permissions in clinical consent forms.<\/jats:p><jats:p>Methods\u2003We conducted a retrospective analysis of clinical consent forms. We developed an annotation scheme following the MAMA (Model-Annotate-Model-Annotate) cycle and evaluated interannotator agreement (IAA) using observed agreement (A o), weighted kappa (\u03baw ), and Krippendorff's \u03b1.<\/jats:p><jats:p>Results\u2003The final dataset included 6,399 sentences from 134 clinical consent forms. Complete agreement was achieved for 5,871 sentences, including 211 positively identified and 5,660 negatively identified as permission-sentences across all three annotators (A o\u2009=\u20090.944, Krippendorff's \u03b1\u2009=\u20090.599). These values reflect moderate to substantial IAA. Although permission-sentences contain a set of common words and structure, disagreements between annotators are largely explained by lexical variability and ambiguity in sentence meaning.<\/jats:p><jats:p>Conclusion\u2003Our findings point to the complexity of identifying permission-sentences within the clinical consent forms. We present our results in light of lessons learned, which may serve as a launching point for developing tools for automated permission extraction.<\/jats:p>","DOI":"10.1055\/s-0041-1730032","type":"journal-article","created":{"date-parts":[[2021,6,23]],"date-time":"2021-06-23T22:55:20Z","timestamp":1624488920000},"page":"429-435","source":"Crossref","is-referenced-by-count":2,"title":["Lessons Learned for Identifying and Annotating Permissions in Clinical Consent Forms"],"prefix":"10.1055","volume":"12","author":[{"given":"Elizabeth E.","family":"Umberfield","sequence":"additional","affiliation":[{"name":"Health Policy & Management, Indiana University Richard M Fairbanks School of Public Health, Indianapolis, Indiana, United States"},{"name":"Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, United States"}]},{"given":"Yun","family":"Jiang","sequence":"additional","affiliation":[{"name":"Department of Systems, Populations and Leadership, University of Michigan School of Nursing, Ann Arbor, Michigan, United States"}]},{"given":"Susan H.","family":"Fenton","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, University of Texas Health Science Center, Houston, Texas, United States"}]},{"given":"Cooper","family":"Stansbury","sequence":"additional","affiliation":[{"name":"Department of Computational Medicine and Bioinformatics, University of Michigan Medical School, Ann Arbor, Michigan, United States"},{"name":"The Michigan Institute for Computational Discovery and Engineering, University of Michigan, Ann Arbor, Michigan, United States"}]},{"given":"Kathleen","family":"Ford","sequence":"additional","affiliation":[{"name":"Department of Systems, Populations and Leadership, University of Michigan School of Nursing, Ann Arbor, Michigan, United States"}]},{"given":"Kaycee","family":"Crist","sequence":"additional","affiliation":[{"name":"Rory Meyers School of Nursing, New York University, New York, New York, United States"}]},{"given":"Sharon L. R.","family":"Kardia","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, University of Michigan School of Public Health, Ann Arbor, Michigan, United States"}]},{"given":"Andrea K.","family":"Thomer","sequence":"additional","affiliation":[{"name":"University of Michigan School of Information, Ann Arbor, Michigan, United States"}]},{"given":"Marcelline R.","family":"Harris","sequence":"additional","affiliation":[{"name":"Department of Systems, Populations and Leadership, University of Michigan School of Nursing, Ann Arbor, Michigan, United States"}]}],"member":"194","published-online":{"date-parts":[[2021,6,23]]},"reference":[{"issue":"01","key":"ref2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1055\/s-0040-1721013","article-title":"Electronic health records in Danish home care and nursing homes: inadequate documentation of care, medication, and consent","volume":"12","author":"M Hertzum","year":"2021","journal-title":"Appl Clin Inform"},{"issue":"09","key":"ref3","doi-asserted-by":"crossref","first-page":"854","DOI":"10.1016\/j.ijmedinf.2013.04.008","article-title":"An investigation of the efficacy of electronic consenting interfaces of research permissions management system in a hospital setting","volume":"82","author":"K Chalil Madathil","year":"2013","journal-title":"Int J Med Inform"},{"issue":"08","key":"ref4","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1001\/jamasurg.2020.1014","article-title":"Association of electronic surgical consent forms with entry error rates","volume":"155","author":"J J Reeves","year":"2020","journal-title":"JAMA Surg"},{"key":"ref5","first-page":"80","article-title":"Replacing paper informed consent with electronic informed consent for research in academic medical centers: a scoping review","volume":"2020","author":"C Chen","year":"2020","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"ref6","first-page":"26, 28, 30, 32","article-title":"Engagement shift: informed consent in the digital era","volume":"25","author":"J Litwin","year":"2016","journal-title":"Appl Clin Trials"},{"key":"ref7","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1145\/1920320.1920322","article-title":"A biomedical research permissions ontology: cognitive and knowledge representation considerations","volume":"2010","author":"J Obeid","year":"2010","journal-title":"Proc Gov Technol Inf Policies"},{"issue":"02","key":"ref9","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.jbi.2005.06.004","article-title":"Inductive creation of an annotation schema for manually indexing clinical conditions from emergency department reports","volume":"39","author":"W W Chapman","year":"2006","journal-title":"J Biomed Inform"},{"key":"ref13","first-page":"21","volume-title":"Designing annotation schemes: from theory to model","author":"J Pustejovsky","year":"2017"},{"key":"ref14","volume-title":"Dataturks","year":"2018"},{"key":"ref16","first-page":"297","volume-title":"Inter-annotator Agreement","author":"R Artstein","year":"2017"},{"issue":"06","key":"ref20","doi-asserted-by":"crossref","first-page":"830","DOI":"10.1111\/cts.12364","article-title":"Readability of invasive procedure consent forms","volume":"8","author":"A EM Eltorai","year":"2015","journal-title":"Clin Transl Sci"},{"issue":"01","key":"ref22","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"J R Landis","year":"1977","journal-title":"Biometrics"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/s-0041-1730032.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,5]],"date-time":"2023-11-05T07:54:26Z","timestamp":1699170866000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0041-1730032"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5]]},"references-count":12,"journal-issue":{"issue":"03","published-online":{"date-parts":[[2021,5,26]]},"published-print":{"date-parts":[[2021,5]]}},"URL":"https:\/\/doi.org\/10.1055\/s-0041-1730032","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5]]}}}