{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T22:48:19Z","timestamp":1759963699974},"reference-count":19,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2014,1,14]],"date-time":"2014-01-14T00:00:00Z","timestamp":1389657600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2014,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>The Centers for Disease Control and Prevention\u2019s <jats:italic>Guidelines for Evaluating Public Health Surveillance Systems<\/jats:italic> are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these <jats:italic>Guidelines,<\/jats:italic> and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers\u2019 immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusions<\/jats:title>\n            <jats:p>Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/1472-6947-14-5","type":"journal-article","created":{"date-parts":[[2014,1,14]],"date-time":"2014-01-14T22:01:55Z","timestamp":1389736915000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Electronic immunization data collection systems: application of an evaluation framework"],"prefix":"10.1186","volume":"14","author":[{"given":"Christine L","family":"Heidebrecht","sequence":"first","affiliation":[]},{"given":"Jeffrey C","family":"Kwong","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Finkelstein","sequence":"additional","affiliation":[]},{"given":"Sherman D","family":"Quan","sequence":"additional","affiliation":[]},{"given":"Jennifer A","family":"Pereira","sequence":"additional","affiliation":[]},{"given":"Susan","family":"Quach","sequence":"additional","affiliation":[]},{"given":"Shelley L","family":"Deeks","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2014,1,14]]},"reference":[{"key":"777_CR1","volume-title":"Influenza","author":"Public Health Agency of Canada","year":"2012","unstructured":"Public Health Agency of Canada: Influenza. 2012,http:\/\/www.phac-aspc.gc.ca\/influenza\/index-eng.php,"},{"key":"777_CR2","doi-asserted-by":"crossref","first-page":"20158","DOI":"10.2807\/ese.17.17.20158-en","volume":"17","author":"JA Laroche","year":"2012","unstructured":"Laroche JA, Diniz AJ: Immunisation registers in Canada: progress made, current situation, and challenges for the future. Euro Surveill. 2012, 17: 20158-","journal-title":"Euro Surveill"},{"key":"777_CR3","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1007\/BF03404175","volume":"102","author":"CL Heidebrecht","year":"2011","unstructured":"Heidebrecht CL, Pereira JA, Quach S, Foisy J, Quan SD, Finkelstein M, Deeks SL, Guay M, Bettinger JA, Kolbe F, Crowcroft NS, Sikora CA, Buckeridge DL, Kwong JC: Approaches to immunization data collection employed across Canada during the pandemic (H1N1) 2009 influenza vaccination campaign. Can J Public Health. 2011, 102: 349-54.","journal-title":"Can J Public Health"},{"issue":"RR-13","key":"777_CR4","first-page":"1","volume":"50","author":"Centers for Disease Control and Prevention","year":"2001","unstructured":"Centers for Disease Control and Prevention: Updated guidelines for evaluating public health surveillance systems. MMWR Recomm Rep. 2001, 50 (RR-13): 1-35.","journal-title":"MMWR Recomm Rep"},{"issue":"1","key":"777_CR5","first-page":"10","volume":"60","author":"Centers for Disease Control and Prevention","year":"2011","unstructured":"Centers for Disease Control and Prevention: Progress in immunization information systems - United States, 2009. MMWR Morb Mortal Wkly Rep. 2011, 60 (1): 10-12.","journal-title":"MMWR Morb Mortal Wkly Rep"},{"issue":"Suppl 6","key":"777_CR6","first-page":"1","volume":"28","author":"H Canada","year":"2002","unstructured":"Canada H: Functional standards and minimum (core) data sets for a national immunization registry network and vaccine associated adverse events surveillance system. Can Commun Dis Rep. 2002, 28 (Suppl 6): 1-32.","journal-title":"Can Commun Dis Rep"},{"key":"777_CR7","volume-title":"The immunization data quality self-assessment (DQS) tool","author":"World Health Organization","year":"2005","unstructured":"World Health Organization: The immunization data quality self-assessment (DQS) tool. 2005,http:\/\/whqlibdoc.who.int\/hq\/2005\/WHO_IVB_05.04.pdf,"},{"key":"777_CR8","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1186\/1471-2288-10-51","volume":"10","author":"JA Pereira","year":"2010","unstructured":"Pereira JA, Quach S, Heidebrecht CL, Foisy J, Quan S, Finkelstein M, Sikora CA, Bettinger JA, Buckeridge DL, McCarthy A, Deeks SL, Kwong JC: Pan-Canadian assessment of pandemic immunization data collection: study methodology. BMC Med Res Methodol. 2010, 10: 51-10.1186\/1471-2288-10-51.","journal-title":"BMC Med Res Methodol"},{"key":"777_CR9","volume-title":"Epidemiology: Beyond the Basics","author":"M Szklo","year":"2007","unstructured":"Szklo M, Nieto FJ: Epidemiology: Beyond the Basics. 2007, Burlington: Jones and Bartlett, 2","edition":"2"},{"key":"777_CR10","unstructured":"Powerpoint Presentation. Ontario Ministry of Health and Long-Term Care: Overview of the Panorama Project in Ontario. 2012, Ontario Public Health Integrated Solutions Branch"},{"issue":"12","key":"777_CR11","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1016\/j.ijmedinf.2008.06.007","volume":"77","author":"AK Jha","year":"2008","unstructured":"Jha AK, Doolan D, Grandt D, Scott T, Bates DW: The use of health information technology in seven nations. Int J Med Inform. 2008, 77 (12): 848-54. 10.1016\/j.ijmedinf.2008.06.007.","journal-title":"Int J Med Inform"},{"issue":"10","key":"777_CR12","doi-asserted-by":"publisher","first-page":"713","DOI":"10.1016\/j.ijmedinf.2012.07.007","volume":"81","author":"CA Salzberg","year":"2012","unstructured":"Salzberg CA, Jang Y, Rozenblum R, Zimlichman E, Tambllyn R, Bates DW: Policy initiatives for health information technology: a qualitative study of U.S. Expectations and Canada\u2019s experience. Int J Med Inform. 2012, 81 (10): 713-22. 10.1016\/j.ijmedinf.2012.07.007.","journal-title":"Int J Med Inform"},{"key":"777_CR13","volume-title":"Building a roadmap for health information systems interoperability for public health","author":"Public Health Data Standards Consortium","year":"2007","unstructured":"Public Health Data Standards Consortium: Building a roadmap for health information systems interoperability for public health. 2007,http:\/\/www.ihe.net\/technical_framework\/upload\/ihe_phdsc_public_health_white_paper_2007_10_11.pdf,"},{"key":"777_CR14","first-page":"575","volume-title":"AMIA Annu Symp Proc","author":"AO Orlova","year":"2005","unstructured":"Orlova AO, Dunnagan M, Finitzo T, Higgins M, Watkins T, Tien A, Beals S: An Electronic Health Record-Public Health (EHR-PH) System Prototype for Interoperability in 21st Century Healthcare Systems. AMIA Annu Symp Proc. 2005, 575-9.http:\/\/www.cdc.gov\/mmwr\/preview\/mmwrhtml\/rr5013a1.htm,"},{"key":"777_CR15","doi-asserted-by":"publisher","first-page":"5054","DOI":"10.1016\/j.vaccine.2009.06.056","volume":"27","author":"BP Hull","year":"2009","unstructured":"Hull BP, Deeks SL, McIntyre PB: The Australian Childhood Immunisation Register - A model for universal immunisation registers?. Vaccine. 2009, 27: 5054-5060. 10.1016\/j.vaccine.2009.06.056.","journal-title":"Vaccine"},{"key":"777_CR16","first-page":"20148","volume":"17","author":"LK Chin","year":"2012","unstructured":"Chin LK, Crawford NW, Rowles G, Butterly JP: Australian immunisation registers: established foundations and opportunities for improvement. Euro Surveill. 2012, 17: 20148-","journal-title":"Euro Surveill"},{"key":"777_CR17","volume-title":"Manitoba Immunization Monitoring System (MIMS) annual report","author":"Manitoba Health","year":"2010","unstructured":"Manitoba Health: Manitoba Immunization Monitoring System (MIMS) annual report. 2010,http:\/\/www.gov.mb.ca\/health\/publichealth\/surveillance\/mims\/reports\/2010.pdf,"},{"key":"777_CR18","doi-asserted-by":"crossref","first-page":"20155","DOI":"10.2807\/ese.17.17.20155-en","volume":"17","author":"T Grove Krause","year":"2012","unstructured":"Grove Krause T, Jakobsen S, Haarh M, M\u00f8lbak K: The Danish vaccination register. Euro Surveill. 2012, 17: 20155-","journal-title":"Euro Surveill"},{"key":"777_CR19","doi-asserted-by":"crossref","first-page":"20147","DOI":"10.2807\/ese.17.16.20147-en","volume":"17","author":"L Trogstad","year":"2012","unstructured":"Trogstad L, Ung G, Hagerup-Jenssen M, Cappelen I, Haugen IL, Feiring B: The Norwegian immunisation register \u2013 SYSVAK. Euro Surveill. 2012, 17: 20147-","journal-title":"Euro Surveill"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/1472-6947-14-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/1472-6947-14-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/1472-6947-14-5","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1472-6947-14-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,2]],"date-time":"2021-09-02T02:48:34Z","timestamp":1630550914000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/1472-6947-14-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,1,14]]},"references-count":19,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,12]]}},"alternative-id":["777"],"URL":"https:\/\/doi.org\/10.1186\/1472-6947-14-5","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,1,14]]},"assertion":[{"value":"25 April 2013","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 December 2013","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 January 2014","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"5"}}