{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T16:50:22Z","timestamp":1791651022834,"version":"4.3.4"},"reference-count":22,"publisher":"Wiley","license":[{"start":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T00:00:00Z","timestamp":1791590400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T00:00:00Z","timestamp":1791590400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01AG073410"],"award-info":[{"award-number":["R01AG073410"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01AG073598"],"award-info":[{"award-number":["R01AG073598"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01HL161253"],"award-info":[{"award-number":["R01HL161253"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01NS102190"],"award-info":[{"award-number":["R01NS102190"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01NS102574"],"award-info":[{"award-number":["R01NS102574"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01NS107291"],"award-info":[{"award-number":["R01NS107291"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01NS126282"],"award-info":[{"award-number":["R01NS126282"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["RF1AG064312"],"award-info":[{"award-number":["RF1AG064312"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["RF1NS120947"],"award-info":[{"award-number":["RF1NS120947"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Epilepsia"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Determining whether an individual reader has sufficient electroencephalographic (EEG) spike\u2010detection skill is challenging due to limited expert agreement. We evaluated whether modeling skill in interpretable signal\u2010detection terms\u2014discrimination ability separate from response tendency\u2014could power an adaptive, variable\u2010length test that can reliably and efficiently assess reader competence.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We analyzed a multirater dataset containing 20\u2009521 EEG segments and 2574 rater sessions. Case signal was estimated from multirater labels, and each reader was modeled using signal\u2010detection parameters: discrimination skill  and response bias . Among 368 calibration\u2010eligible raters across five skill tiers, we used a stratified 50\/50 train\u2013validation split to select a certification threshold  in training and to apply it in validation. We replayed an adaptive test on each rater's historical responses, using sequential stopping to issue pass, fail, or retest decisions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The training\u2010derived threshold was  (95% bootstrap confidence interval = \u2013, Youden ). In validation, pass rates preserved the expected tier ordering: experts 100% (), experienced 100% (), novices 14% (), residents 0% (), and crowd raters 7% (). In the 81\u2010rater adaptive replay, clear failures stopped near 20 trials, and passing experts required a median of 95 trials. Split\u2010half replay showed intraclass correlation coefficient  for  at stopping; there were no hard pass\u2013fail reversals among raters whose two halves both reached decisive verdicts.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Significance<\/jats:title>\n                    <jats:p>A signal\u2010detection framework quantified spike\u2010identification skill in EEG readers, and adaptive testing estimated this skill reliably with fewer than 100 short EEG segments. This approach could provide a practical, scalable method for assessing EEG reading competence.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/epi.70519","type":"journal-article","created":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T16:05:36Z","timestamp":1791648336000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["An adaptive, variable\u2010length test for certifying electroencephalographic spike\u2010detection skill: A retrospective feasibility study"],"prefix":"10.1002","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-2621-9929","authenticated-orcid":false,"given":"Tianyu","family":"Zhang","sequence":"first","affiliation":[{"name":"Beth Israel Deaconess Medical Center  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7161-6385","authenticated-orcid":false,"given":"F\u00e1bio A.","family":"Nascimento","sequence":"additional","affiliation":[{"name":"Washington University School of Medicine  St. Louis Missouri USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5122-7211","authenticated-orcid":false,"given":"Ioannis","family":"Karakis","sequence":"additional","affiliation":[{"name":"Emory University School of Medicine  Atlanta Georgia USA"},{"name":"Medical School of the University of Crete  Heraklion Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4828-8629","authenticated-orcid":false,"given":"Elijah W.","family":"Keldsen","sequence":"additional","affiliation":[{"name":"Stanford University School of Medicine  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1762-0877","authenticated-orcid":false,"given":"ChenXi","family":"Sun","sequence":"additional","affiliation":[{"name":"Stanford University School of Medicine  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2415-5854","authenticated-orcid":false,"given":"Jin","family":"Jing","sequence":"additional","affiliation":[{"name":"Beth Israel Deaconess Medical Center  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4803-312X","authenticated-orcid":false,"given":"M. Brandon","family":"Westover","sequence":"additional","affiliation":[{"name":"Stanford University School of Medicine  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,10,10]]},"reference":[{"issue":"9","key":"e_1_2_13_2_1","doi-asserted-by":"crossref","first-page":"1661","DOI":"10.1016\/j.clinph.2014.11.008","article-title":"Inter\u2010rater agreement on identification of electrographic seizures and periodic discharges in ICU EEG recordings","volume":"126","author":"Halford JJ","year":"2015","journal-title":"Clin Neurophysiol"},{"issue":"2","key":"e_1_2_13_3_1","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1097\/WNP.0000000000000344","article-title":"Characteristics of EEG interpreters associated with higher interrater agreement","volume":"34","author":"Halford JJ","year":"2017","journal-title":"J Clin Neurophysiol"},{"issue":"3","key":"e_1_2_13_4_1","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1016\/j.clinph.2014.05.037","article-title":"The probability of seizures during EEG monitoring in critically ill adults","volume":"126","author":"Westover MB","year":"2015","journal-title":"Clin Neurophysiol"},{"key":"e_1_2_13_5_1","doi-asserted-by":"crossref","DOI":"10.1016\/j.jneumeth.2020.108956","article-title":"Rapid annotation of seizures and interictal\u2013ictal\u2013injury continuum EEG patterns","volume":"347","author":"Jing J","year":"2021","journal-title":"J Neurosci Methods"},{"issue":"17","key":"e_1_2_13_6_1","first-page":"e1750","article-title":"Development of expert\u2010level classification of seizures and rhythmic and periodic patterns during EEG interpretation","volume":"100","author":"Jing J","year":"2023","journal-title":"Neurology"},{"issue":"9","key":"e_1_2_13_7_1","doi-asserted-by":"crossref","first-page":"1366","DOI":"10.1111\/epi.12653","article-title":"Interrater agreement for critical care EEG terminology","volume":"55","author":"Gaspard N","year":"2014","journal-title":"Epilepsia"},{"issue":"1","key":"e_1_2_13_8_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1097\/WNP.0000000000000806","article-title":"American clinical neurophysiology Society's standardized critical care EEG terminology: 2021 version","volume":"38","author":"Hirsch LJ","year":"2021","journal-title":"J Clin Neurophysiol"},{"issue":"9","key":"e_1_2_13_9_1","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1097\/00004424-199209000-00015","article-title":"Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method","volume":"27","author":"Dorfman DD","year":"1992","journal-title":"Investig Radiol"},{"issue":"2","key":"e_1_2_13_10_1","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1080\/03610919508813243","article-title":"Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an ANOVA approach with dependent observations","volume":"24","author":"Obuchowski NA","year":"1995","journal-title":"Commun Stat Simul Comput"},{"key":"e_1_2_13_11_1","volume-title":"Signal detection theory and psychophysics","author":"Green DM","year":"1966"},{"key":"e_1_2_13_12_1","volume-title":"Detection theory: a user's guide","author":"Macmillan NA","year":"2005"},{"key":"e_1_2_13_13_1","volume-title":"Probabilistic models for some intelligence and attainment tests","author":"Rasch G","year":"1960"},{"key":"e_1_2_13_14_1","doi-asserted-by":"crossref","DOI":"10.4324\/9781410605931","volume-title":"Computerized adaptive testing: a primer","author":"Wainer H","year":"2000"},{"key":"e_1_2_13_15_1","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-85461-8","volume-title":"Elements of adaptive testing","author":"Linden WJ","year":"2010"},{"issue":"3","key":"e_1_2_13_16_1","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1167\/17.3.10","article-title":"QUEST+: a general multidimensional Bayesian adaptive psychometric method","volume":"17","author":"Watson AB","year":"2017","journal-title":"J Vis"},{"issue":"2","key":"e_1_2_13_17_1","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1214\/aoms\/1177731118","article-title":"Sequential tests of statistical hypotheses","volume":"16","author":"Wald A","year":"1945","journal-title":"Ann Math Stat"},{"key":"e_1_2_13_18_1","first-page":"237","volume-title":"New horizons in testing: latent trait test theory and computerized adaptive testing","author":"Reckase MD","year":"1983"},{"issue":"4","key":"e_1_2_13_19_1","doi-asserted-by":"crossref","first-page":"405","DOI":"10.3102\/10769986021004405","article-title":"Comparison of SPRT and sequential bayes procedures for classifying examinees into two categories using a computerized test","volume":"21","author":"Spray JA","year":"1996","journal-title":"J Educ Behav Stat"},{"issue":"4","key":"e_1_2_13_20_1","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1177\/014662169001400404","article-title":"Using Bayesian decision theory to design a computerized mastery test","volume":"14","author":"Lewis C","year":"1990","journal-title":"Appl Psychol Meas"},{"key":"e_1_2_13_21_1","first-page":"257","volume-title":"New horizons in testing: latent trait test theory and computerized adaptive testing","author":"Kingsbury GG","year":"1983"},{"key":"e_1_2_13_22_1","doi-asserted-by":"crossref","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins GS","year":"2024","journal-title":"BMJ"},{"issue":"4","key":"e_1_2_13_23_1","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1002\/epd2.20229","article-title":"A randomized controlled educational pilot trial of interictal epileptiform discharge identification for neurology residents","volume":"26","author":"Nascimento FA","year":"2024","journal-title":"Epileptic Disord"}],"container-title":["Epilepsia"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/epi.70519","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/epi.70519","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/epi.70519","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T16:05:53Z","timestamp":1791648353000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/epi.70519"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,10]]},"references-count":22,"alternative-id":["10.1002\/epi.70519"],"URL":"https:\/\/doi.org\/10.1002\/epi.70519","archive":["Portico"],"relation":{},"ISSN":["0013-9580","1528-1167"],"issn-type":[{"value":"0013-9580","type":"print"},{"value":"1528-1167","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,10]]},"assertion":[{"value":"2026-06-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-17","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-10-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"epi.70519"}}