{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T07:54:12Z","timestamp":1777362852615,"version":"3.51.4"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643683881","type":"print"},{"value":"9781643683898","type":"electronic"}],"license":[{"start":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T00:00:00Z","timestamp":1684368000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,5,18]]},"abstract":"<jats:p>Risk of bias (RoB) assessment of randomized clinical trials (RCTs) is vital to conducting systematic reviews. Manual RoB assessment for hundreds of RCTs is a cognitively demanding, lengthy process and is prone to subjective judgment. Supervised machine learning (ML) can help to accelerate this process but requires a hand-labelled corpus. There are currently no RoB annotation guidelines for randomized clinical trials or annotated corpora. In this pilot project, we test the practicality of directly using the revised Cochrane RoB 2.0 guidelines for developing an RoB annotated corpus using a novel multi-level annotation scheme. We report inter-annotator agreement among four annotators who used Cochrane RoB 2.0 guidelines. The agreement ranges between 0% for some bias classes and 76% for others. Finally, we discuss the shortcomings of this direct translation of annotation guidelines and scheme and suggest approaches to improve them to obtain an RoB annotated corpus suitable for ML.<\/jats:p>","DOI":"10.3233\/shti230210","type":"book-chapter","created":{"date-parts":[[2023,5,19]],"date-time":"2023-05-19T04:45:36Z","timestamp":1684471536000},"source":"Crossref","is-referenced-by-count":1,"title":["First Steps Towards a Risk of Bias Corpus of Randomized Controlled Trials"],"prefix":"10.3233","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1691-1338","authenticated-orcid":false,"given":"Anjani","family":"Dhrangadhariya","sequence":"first","affiliation":[{"name":"Informatics Institute, HES-SO Valais-Wallis, Sierre, Switzerland"},{"name":"University of Geneva (UNIGE), Geneva, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roger","family":"Hilfiker","sequence":"additional","affiliation":[{"name":"IUFRS, University of Lausanne, Lausanne, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin","family":"Sattelmayer","sequence":"additional","affiliation":[{"name":"School of Health Sciences, HES-SO Valais-Wallis, Leukerbad, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katia","family":"Giacomino","sequence":"additional","affiliation":[{"name":"School of Health Sciences, HES-SO Valais-Wallis, Leukerbad, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rahel","family":"Caliesch","sequence":"additional","affiliation":[{"name":"School of Health Sciences, HES-SO Valais-Wallis, Leukerbad, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simone","family":"Elsig","sequence":"additional","affiliation":[{"name":"School of Health Sciences, HES-SO Valais-Wallis, Leukerbad, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nona","family":"Naderi","sequence":"additional","affiliation":[{"name":"Geneva School of Business Administration, HES-SO Geneva, Switzerland"},{"name":"SIB Swiss Institute of Bioinformatics (SIB), Geneva, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Henning","family":"M\u00fcller","sequence":"additional","affiliation":[{"name":"Informatics Institute, HES-SO Valais-Wallis, Sierre, Switzerland"},{"name":"University of Geneva (UNIGE), Geneva, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Studies in Health Technology and Informatics","Caring is Sharing \u2013 Exploiting the Value in Data for Health and Innovation"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/SHTI230210","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T10:59:26Z","timestamp":1685530766000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI230210"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,18]]},"ISBN":["9781643683881","9781643683898"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/shti230210","relation":{},"ISSN":["0926-9630","1879-8365"],"issn-type":[{"value":"0926-9630","type":"print"},{"value":"1879-8365","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,18]]}}}