{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T04:08:52Z","timestamp":1776830932354,"version":"3.51.2"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2023,6,16]],"date-time":"2023-06-16T00:00:00Z","timestamp":1686873600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0"},{"start":{"date-parts":[[2023,6,16]],"date-time":"2023-06-16T00:00:00Z","timestamp":1686873600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0"}],"funder":[{"DOI":"10.13039\/100000002","name":"US National Institutes of Health","doi-asserted-by":"crossref","award":["R01AI153044"],"award-info":[{"award-number":["R01AI153044"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Drug Saf"],"published-print":{"date-parts":[[2023,8]]},"DOI":"10.1007\/s40264-023-01324-1","type":"journal-article","created":{"date-parts":[[2023,6,16]],"date-time":"2023-06-16T17:01:50Z","timestamp":1686934910000},"page":"797-807","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Serially Combining Epidemiological Designs Does Not Improve Overall Signal Detection in Vaccine Safety Surveillance"],"prefix":"10.1007","volume":"46","author":[{"given":"Faaizah","family":"Arshad","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martijn J.","family":"Schuemie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fan","family":"Bu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Evan P.","family":"Minty","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thamir M.","family":"Alshammari","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lana Y. H.","family":"Lai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Talita","family":"Duarte-Salles","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stephen","family":"Fortin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fredrik","family":"Nyberg","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Patrick B.","family":"Ryan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"George","family":"Hripcsak","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Prieto-Alhambra","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9818-479X","authenticated-orcid":false,"given":"Marc A.","family":"Suchard","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,6,16]]},"reference":[{"key":"1324_CR1","unstructured":"Bloomberg [Internet]. https:\/\/www.bloomberg.com\/graphics\/covid-vaccine-tracker-global-distribution\/ [cited 18 Nov 2021]."},{"key":"1324_CR2","unstructured":"WHO Coronavirus (COVID-19) dashboard [Internet]. https:\/\/covid19.who.int [cited 14 Dec 2022]."},{"key":"1324_CR3","unstructured":"CDC. COVID Data Tracker [Internet]; 2020. https:\/\/covid.cdc.gov\/covid-data-tracker\/ [cited 28 July 2021]."},{"key":"1324_CR4","unstructured":"Holder J. Tracking Coronavirus Vaccinations Around the World. The New York Times [Internet]; 2021 Jan 29. https:\/\/www.nytimes.com\/interactive\/2021\/world\/covid-vaccinations-tracker.html [cited 16 Aug 2021]."},{"key":"1324_CR5","unstructured":"COVID-19 Vaccine Tracker [Internet]. https:\/\/vaccinetracker.ecdc.europa.eu\/public\/extensions\/COVID-19\/vaccine-tracker.html [cited 28 July 2021]."},{"issue":"34","key":"1324_CR6","doi-asserted-by":"publisher","DOI":"10.3346\/jkms.2018.33.e213","volume":"33","author":"HS Kim","year":"2018","unstructured":"Kim HS, Lee S, Kim JH. Real-world evidence versus randomized controlled trial: clinical research based on electronic medical records. J Korean Med Sci. 2018;33(34): e213.","journal-title":"J Korean Med Sci"},{"key":"1324_CR7","unstructured":"[Internet]. https:\/\/cioms.ch\/wp-content\/uploads\/2018\/03\/WG8-Signal-Detection.pdf [cited 1 May 2023]."},{"issue":"1","key":"1324_CR8","doi-asserted-by":"publisher","first-page":"66","DOI":"10.2174\/1874844901502010066","volume":"2","author":"A Kumar","year":"2015","unstructured":"Kumar A, Khan H. Signal detection and their assessment in pharmacovigilance. Open Pharm Sci J. 2015;2(1):66\u201373.","journal-title":"Open Pharm Sci J."},{"issue":"10","key":"1324_CR9","doi-asserted-by":"publisher","first-page":"895","DOI":"10.1007\/s40264-015-0314-8","volume":"38","author":"Y Li","year":"2015","unstructured":"Li Y, Ryan PB, Wei Y, Friedman C. A method to combine signals from spontaneous reporting systems and observational healthcare data to detect adverse drug reactions. Drug Saf. 2015;38(10):895.","journal-title":"Drug Saf"},{"issue":"4","key":"1324_CR10","doi-asserted-by":"publisher","DOI":"10.1136\/bmjgh-2018-001065","volume":"4","author":"YM Mesfin","year":"2019","unstructured":"Mesfin YM, Cheng A, Lawrie J, Buttery J. Use of routinely collected electronic healthcare data for postlicensure vaccine safety signal detection: a systematic review. BMJ Glob Health. 2019;4(4): e001065.","journal-title":"BMJ Glob Health"},{"key":"1324_CR11","unstructured":"https:\/\/www.brookings.edu\/wp-content\/uploads\/2012\/04\/Setting-Priorities_Discussion-guide_20110527.pdf [cited 1 May 2023]."},{"key":"1324_CR12","unstructured":"Vaccine Safety Datalink (VSD) [Internet]; 2022. https:\/\/www.cdc.gov\/vaccinesafety\/ensuringsafety\/monitoring\/vsd\/index.html [cited 1 May 2023]."},{"key":"1324_CR13","unstructured":"[Internet]. https:\/\/www.fda.gov\/media\/71238\/download [cited 1 May 2023]."},{"issue":"13","key":"1324_CR14","doi-asserted-by":"publisher","first-page":"811","DOI":"10.3109\/08958378.2014.955932","volume":"26","author":"LD Maxim","year":"2014","unstructured":"Maxim LD, Niebo R, Utell MJ. Screening tests: a review with examples. Inhal Toxicol. 2014;26(13):811\u201328.","journal-title":"Inhal Toxicol"},{"issue":"2","key":"1324_CR15","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1093\/biostatistics\/1.2.123","volume":"1","author":"MS Pepe","year":"2000","unstructured":"Pepe MS, Thompson ML. Combining diagnostic test results to increase accuracy. Biostatistics. 2000;1(2):123\u201340.","journal-title":"Biostatistics"},{"issue":"3","key":"1324_CR16","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1093\/biostatistics\/4.3.341","volume":"4","author":"ML Thompson","year":"2003","unstructured":"Thompson ML. Assessing the diagnostic accuracy of a sequence of tests. Biostatistics. 2003;4(3):341\u201351.","journal-title":"Biostatistics"},{"issue":"13","key":"1324_CR17","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2022.893484","volume":"6","author":"MJ Schuemie","year":"2022","unstructured":"Schuemie MJ, Arshad F, Pratt N, Nyberg F, Alshammari TM, Hripcsak G, et al. Vaccine safety surveillance using routinely collected healthcare data-an empirical evaluation of epidemiological designs. Front Pharmacol. 2022;6(13): 893484.","journal-title":"Front Pharmacol"},{"key":"1324_CR18","unstructured":"Protocol.utf8 [Internet]. https:\/\/ohdsi-studies.github.io\/Eumaeus\/Protocol.html [cited 30 Sep 2021]."},{"issue":"12","key":"1324_CR19","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2021.773875","volume":"24","author":"X Li","year":"2021","unstructured":"Li X, Lai LY, Ostropolets A, Arshad F, Tan EH, Casajust P, et al. Bias, precision and timeliness of historical (background) rate comparison methods for vaccine safety monitoring: an empirical multi-database analysis. Front Pharmacol. 2021;24(12): 773875.","journal-title":"Front Pharmacol"},{"issue":"Suppl 1","key":"1324_CR20","doi-asserted-by":"publisher","first-page":"S54","DOI":"10.1542\/peds.2010-1722I","volume":"127","author":"WK Yih","year":"2011","unstructured":"Yih WK, Kulldorff M, Fireman BH, Shui IM, Lewis EM, Klein NP, et al. Active surveillance for adverse events: the experience of the Vaccine Safety Datalink project. Pediatrics. 2011;127(Suppl 1):S54\u201364.","journal-title":"Pediatrics"},{"key":"1324_CR21","doi-asserted-by":"publisher","first-page":"1768","DOI":"10.1002\/sim.2302","volume":"25","author":"HJ Whitaker","year":"2006","unstructured":"Whitaker HJ, Paddy Farrington C, Spiessens B, Musonda P. Tutorial in biostatistics: the self-controlled case series method [Internet]. Stat Med. 2006;25:1768\u201397. https:\/\/doi.org\/10.1002\/sim.2302.","journal-title":"Stat Med"},{"key":"1324_CR22","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.i4515","volume":"354","author":"I Petersen","year":"2016","unstructured":"Petersen I, Douglas I, Whitaker H. Self controlled case series methods: an alternative to standard epidemiological study designs [Internet]. BMJ. 2016;354: i4515. https:\/\/doi.org\/10.1136\/bmj.i4515.","journal-title":"BMJ"},{"issue":"12","key":"1324_CR23","doi-asserted-by":"publisher","first-page":"1805","DOI":"10.1017\/S0950268811001531","volume":"139","author":"YG Weldeselassie","year":"2011","unstructured":"Weldeselassie YG, Whitaker HJ, Farrington CP. Use of the self-controlled case-series method in vaccine safety studies: review and recommendations for best practice. Epidemiol Infect. 2011;139(12):1805\u201317.","journal-title":"Epidemiol Infect"},{"issue":"2","key":"1324_CR24","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1002\/sim.5925","volume":"33","author":"MJ Schuemie","year":"2014","unstructured":"Schuemie MJ, Ryan PB, DuMouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. Stat Med. 2014;33(2):209\u201318.","journal-title":"Stat Med"},{"issue":"11","key":"1324_CR25","doi-asserted-by":"publisher","first-page":"2571","DOI":"10.1073\/pnas.1708282114","volume":"115","author":"MJ Schuemie","year":"2018","unstructured":"Schuemie MJ, Hripcsak G, Ryan PB, Madigan D, Suchard MA. Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data. Proc Natl Acad Sci USA. 2018;115(11):2571\u20137.","journal-title":"Proc Natl Acad Sci USA"},{"issue":"22","key":"1324_CR26","doi-asserted-by":"publisher","first-page":"3883","DOI":"10.1002\/sim.6977","volume":"35","author":"MJ Schuemie","year":"2016","unstructured":"Schuemie MJ, Hripcsak G, Ryan PB, Madigan D, Suchard MA. Robust empirical calibration of p-values using observational data. Stat Med. 2016;35(22):3883\u20138.","journal-title":"Stat Med"},{"issue":"8","key":"1324_CR27","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0020124","volume":"2","author":"JPA Ioannidis","year":"2005","unstructured":"Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8): e124.","journal-title":"PLoS Med"},{"issue":"8","key":"1324_CR28","doi-asserted-by":"publisher","first-page":"1331","DOI":"10.1093\/jamia\/ocaa103","volume":"27","author":"MJ Schuemie","year":"2020","unstructured":"Schuemie MJ, Ryan PB, Pratt N, Chen R, You SC, Krumholz HM, et al. Principles of large-scale evidence generation and evaluation across a network of databases (LEGEND). J Am Med Inform Assoc. 2020;27(8):1331\u20137.","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"1324_CR29","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1093\/aje\/kws317","volume":"177","author":"JC Nelson","year":"2013","unstructured":"Nelson JC, Yu O, Dominguez-Islas CP, Cook AJ, Peterson D, Greene SK, et al. Adapting group sequential methods to observational postlicensure vaccine safety surveillance: results of a pentavalent combination DTaP-IPV-Hib vaccine safety study. Am J Epidemiol. 2013;177(2):131\u201341.","journal-title":"Am J Epidemiol"},{"issue":"8","key":"1324_CR30","doi-asserted-by":"publisher","first-page":"839","DOI":"10.1002\/pds.3618","volume":"23","author":"JC Maro","year":"2014","unstructured":"Maro JC, Brown JS, Dal Pan GJ, Kulldorff M. Minimizing signal detection time in postmarket sequential analysis: balancing positive predictive value and sensitivity. Pharmacoepidemiol Drug Saf. 2014;23(8):839\u201348.","journal-title":"Pharmacoepidemiol Drug Saf"},{"issue":"8","key":"1324_CR31","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1186\/s13104-015-1454-8","volume":"12","author":"II Mbachu","year":"2015","unstructured":"Mbachu II, Udigwe G, Joseph I, John O, Samuel UO, Joseph U, et al. The evaluation of accuracy of serial rapid HIV test algorithm in the diagnosis of HIV antibodies among pregnant women in south east Nigeria. BMC Res Notes. 2015;12(8):557.","journal-title":"BMC Res Notes"},{"key":"1324_CR32","doi-asserted-by":"crossref","unstructured":"Koblavi-D\u00e8me S, Maurice C, Yavo D, Sibailly TS, N\u2019guessan K, Kamelan-Tano Y, et al. Sensitivity and specificity of human immunodeficiency virus rapid serologic assays and testing algorithms in an antenatal clinic in Abidjan Ivory Coast. J Clin Microbiol. 2001;39(5):1808\u201312.","DOI":"10.1128\/JCM.39.5.1808-1812.2001"},{"issue":"1\u20132","key":"1324_CR33","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.jviromet.2013.04.003","volume":"192","author":"RM Galiwango","year":"2013","unstructured":"Galiwango RM, Musoke R, Lubyayi L, Ssekubugu R, Kalibbala S, Ssekweyama V, et al. Evaluation of current rapid HIV test algorithms in Rakai, Uganda. J Virol Methods. 2013;192(1\u20132):25\u20137.","journal-title":"J Virol Methods"},{"key":"1324_CR34","doi-asserted-by":"crossref","unstructured":"Iqbal HS, Solomon S, Murugavel KG, Solomon SS, Balakrishnan P. Evaluation of two indigenous rapid and two ELISA assays for the diagnosis of HIV infection India. Indian J Med Microbiol [Internet]. 2012;30(4). https:\/\/pubmed.ncbi.nlm.nih.gov\/23183462\/ [cited 17 Dec 2021].","DOI":"10.4103\/0255-0857.103758"},{"issue":"1","key":"1324_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/srep30781","volume":"6","author":"MW Moses","year":"2016","unstructured":"Moses MW, Zwerling A, Cattamanchi A, Denkinger CM, Banaei N, Kik SV, et al. Serial testing for latent tuberculosis using QuantiFERON-TB Gold In-Tube: a Markov model. Sci Rep. 2016;6(1):1\u20138.","journal-title":"Sci Rep"},{"issue":"16","key":"1324_CR36","doi-asserted-by":"publisher","first-page":"1309","DOI":"10.1001\/jama.1996.03540160031029","volume":"276","author":"DS Smith","year":"1996","unstructured":"Smith DS, Catalona WJ, Herschman JD. Longitudinal screening for prostate cancer with prostate-specific antigen. JAMA. 1996;276(16):1309\u201315.","journal-title":"JAMA"},{"issue":"3","key":"1324_CR37","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2018-026012","volume":"9","author":"Z Zhelev","year":"2019","unstructured":"Zhelev Z, Ohtake H, Iwata M, Terasawa T, Rogers M, Peters JL, et al. Diagnostic accuracy of contemporary and high-sensitivity cardiac troponin assays used in serial testing, versus single-sample testing as a comparator, to triage patients suspected of acute non-ST-segment elevation myocardial infarction: a systematic review protocol. BMJ Open. 2019;9(3): e026012.","journal-title":"BMJ Open"},{"issue":"10","key":"1324_CR38","doi-asserted-by":"publisher","first-page":"2092","DOI":"10.1002\/1097-0142(19951115)76:10+<2092::AID-CNCR2820761331>3.0.CO;2-T","volume":"76","author":"JS Berek","year":"1995","unstructured":"Berek JS, Bast RC Jr. Ovarian cancer screening. The use of serial complementary tumor markers to improve sensitivity and specificity for early detection. Cancer. 1995;76(10):2092\u20136.","journal-title":"Cancer"},{"issue":"9","key":"1324_CR39","doi-asserted-by":"crossref","DOI":"10.1093\/cid\/ciaa1630","volume":"73","author":"T Kobayashi","year":"2020","unstructured":"Kobayashi T, Trannel A, Holley SA, Alsuhaibani MA, Abosi OJ, Jenn KE, et al. Coronavirus disease 2019 serial testing among hospitalized patients in a Midwest Tertiary Medical Center, July\u2013September 2020. Clin Infect Dis. 2020;73(9): e3116\u20139.","journal-title":"Clin Infect Dis"},{"issue":"1","key":"1324_CR40","first-page":"1219","volume":"4","author":"JC Nelson","year":"2016","unstructured":"Nelson JC, Wellman R, Yu O, Cook AJ, Maro JC, Ouellet-Hellstrom R, et al. A synthesis of current surveillance planning methods for the sequential monitoring of drug and vaccine adverse effects using electronic health care data. EGEMS (Wash DC). 2016;4(1):1219.","journal-title":"EGEMS (Wash DC)"}],"container-title":["Drug Safety"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40264-023-01324-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40264-023-01324-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40264-023-01324-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T20:09:15Z","timestamp":1709928555000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40264-023-01324-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,16]]},"references-count":40,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["1324"],"URL":"https:\/\/doi.org\/10.1007\/s40264-023-01324-1","relation":{},"ISSN":["0114-5916","1179-1942"],"issn-type":[{"value":"0114-5916","type":"print"},{"value":"1179-1942","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,16]]},"assertion":[{"value":"29 May 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 June 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This work was supported by US National Institutes of Health [grant number R01AI153044], a contract from the US Food and Drug Administration and an Intergovernmental Personnel Act agreement with the US Department of Veterans Affairs.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Funding"}},{"value":"FA and FB declare no competing interests. MAS receives grant funding from the US National Institutes of Health and the US Food and Drug Administration and contracts from the US Department of Veterans Affairs and Janssen Research and Development. PBR, SF and MJS are employees of Janssen Research and Development and shareholders in Johnson & Johnson. GH receives grant funding from the US National Institutes of Health and the US Food and Drug Administration. DPA\u2019s research group has received grant support from Amgen, Chesi-Taylor, Novartis, and UCB Biopharma. His department has received advisory or consultancy fees from Amgen, Astellas, AstraZeneca, Johnson, and Johnson, and UCB Biopharma and fees for speaker services from Amgen and UCB Biopharma. Janssen, on behalf of IMI-funded EHDEN and EMIF consortiums, and Synapse Management Partners have supported training programmes organized by DPA's department and open for external participants organized by his department outside submitted work. FN was an employee of AstraZeneca until 2019 and owns some AstraZeneca shares. EPM, LYHL, TMA, and TDS declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}},{"value":"This study did not require Institutional Review Board approval from data partners because it used de-identified patient data and did not constitute research with human subjects.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"Not applicable.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"The data supporting the findings of this study are available at .","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Data Availability"}},{"value":"The protocol and analytical code underlying this manuscript are available in the Eumaeus Repository at .","order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code Availability"}},{"value":"FA, MJS, PBR, GH, DPA and MAS conceived the research. FA, MJS and MAS drafted the manuscript in consultation with EPM, TMA, LYHL, TDS, SF, FN, FB, PBR, GH, and DPA, who provided critical feedback on the research and writing. All authors read and approved the final version.","order":9,"name":"Ethics","group":{"name":"EthicsHeading","label":"Author Contributions"}}]}}