{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T22:39:54Z","timestamp":1779403194710,"version":"3.53.1"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643686615","type":"electronic"}],"license":[{"start":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T00:00:00Z","timestamp":1779321600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,5,21]]},"abstract":"<jats:p>Systematic reviews are essential for evidence-based healthcare but remain highly resource-intensive, with most retrieved studies ultimately excluded after manual screening. This study developed and evaluated a hybrid expert\u2013LLM workflow to reduce human workload while maintaining accuracy and transparency. Within the Thyroid Risk Stratification Tool (ThyRST) project, ChatGPT-5 was used to classify 14,858 records on thyroid nodule malignancy risk into thematic categories. Recurrent irrelevant concepts were refined through expert consensus involving clinicians and informaticians and embedded as exclusion rules in structured prompts. The model then labelled each abstract as INCLUDE, EXCLUDE, or MAYBE, producing outputs for audit and verification. A random sample of 100 records was independently reviewed by human assessors to evaluate performance. The workflow achieved 96% concordance (\u03ba = 0.91) with human reviewers, with only one false exclusion, and reduced manual screening time by approximately 70%. These results demonstrate that a transparent Delphi-inspired expert\u2013LLM can accurately and reproducibly automate early-stage evidence screening, providing substantial efficiency gains while preserving human oversight and methodological rigor. The approach offers a practical pathway toward the responsible integration of generative AI in systematic review methodology and digital health research.<\/jats:p>","DOI":"10.3233\/shti260255","type":"book-chapter","created":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T21:59:31Z","timestamp":1779400771000},"source":"Crossref","is-referenced-by-count":0,"title":["A Hybrid Delphi-Inspired Expert\u2013LLM Workflow for Efficient Evidence Screening in Systematic Reviews"],"prefix":"10.3233","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7938-0269","authenticated-orcid":false,"given":"Omid","family":"Pournik","sequence":"first","affiliation":[{"name":"Department of Electronic, Electrical and Systems Engineering, University of Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7087-7324","authenticated-orcid":false,"given":"Emma","family":"Watts","sequence":"additional","affiliation":[{"name":"Department of Applied Health Sciences, University of Birmingham, UK"},{"name":"Department of Otolaryngology and Head and Neck Surgery, University Hospitals Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9778-534X","authenticated-orcid":false,"given":"Emma","family":"Richards","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology and Head and Neck Surgery, University Hospitals Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6435-177X","authenticated-orcid":false,"given":"Kristien","family":"Boelaert","sequence":"additional","affiliation":[{"name":"Department of Applied Health Sciences, University of Birmingham, UK"},{"name":"Department of Endocrinology, University Hospitals Birmingham"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6776-0977","authenticated-orcid":false,"given":"Neil","family":"Sharma","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology and Head and Neck Surgery, University Hospitals Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9814-3023","authenticated-orcid":false,"given":"Saadullah Farooq","family":"Abbasi","sequence":"additional","affiliation":[{"name":"Department of Electronic, Electrical and Systems Engineering, University of Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3073-9514","authenticated-orcid":false,"given":"Xuefei","family":"Ding","sequence":"additional","affiliation":[{"name":"Department of Electronic, Electrical and Systems Engineering, University of Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3690-2359","authenticated-orcid":false,"given":"Leila","family":"Ghalichi","sequence":"additional","affiliation":[{"name":"Department of Applied Health Sciences, University of Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5473-135X","authenticated-orcid":false,"given":"Theodoros N.","family":"Arvanitis","sequence":"additional","affiliation":[{"name":"Department of Electronic, Electrical and Systems Engineering, University of Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"7437","container-title":["Studies in Health Technology and Informatics","Opening the Personal Gate between Technology and Health Care"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/SHTI260255","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T21:59:32Z","timestamp":1779400772000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI260255"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,21]]},"ISBN":["9781643686615"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/shti260255","relation":{},"ISSN":["0926-9630","1879-8365"],"issn-type":[{"value":"0926-9630","type":"print"},{"value":"1879-8365","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,21]]}}}