{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T05:19:45Z","timestamp":1771478385374,"version":"3.50.1"},"reference-count":10,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2013,3,20]],"date-time":"2013-03-20T00:00:00Z","timestamp":1363737600000},"content-version":"vor","delay-in-days":19,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Academic Emergency Medicine"],"published-print":{"date-parts":[[2013,3]]},"abstract":"Abstract<\/jats:title>Background<\/jats:title>The lumbar puncture (LP<\/jats:styled-content>) is a procedural competency deemed necessary by the Accreditation Council for Graduate Medical Education and the Emergency Medicine and Pediatric Residency Review Committees. The emergency department (ED<\/jats:styled-content>) is a primary site for residents to be evaluated performing neonatal LP<\/jats:styled-content>s. Current evaluation methods lack validity evidence as assessment tools.<\/jats:p><\/jats:sec>Objectives<\/jats:title>This was a pilot study to develop an objective structured assessment of technical skills for neonatal LP<\/jats:styled-content> (OSATS<\/jats:styled-content>\u2010LP<\/jats:styled-content>) and to document validity evidence for the instrument in regard to five sources of test validity: content, response process, relation to other variables, inter\u2010rater reliability, and consequences of testing.<\/jats:p><\/jats:sec>Methods<\/jats:title>Pediatric residents were videotaped in the fall of 2011 for comparison of faculty evaluation of resident performance during a neonatal LP<\/jats:styled-content> using a video\u2010delayed format. Residents completed a demographic experience survey evaluating relations to other variables. Content and response process validity was obtained through expert panel meetings and resulted in the following seven domains of performance for the OSATS<\/jats:styled-content>\u2010LP<\/jats:styled-content>: preparation, positioning, analgesia, needle insertion, cerebrospinal fluid (CSF<\/jats:styled-content>) collection, management of laboratory studies, and sterility. t\u2010tests assessed significance between level of training, previous intensive care unit experience, and residents' self\u2010assessed confidence in comparison with their total performance score. The inter\u2010rater agreement of the OSATS<\/jats:styled-content>\u2010LP<\/jats:styled-content> was obtained using the Fleiss' kappa for each domain.<\/jats:p><\/jats:sec>Results<\/jats:title>Sixteen pediatric residents completed the simulation with six raters evaluating each resident (96 ratings). The domains of sterility and CSF<\/jats:styled-content> collection had moderate statistical reliability (\u03ba\u00a0=\u00a00.41 and 0.51, respectively). The domains of preparation, analgesia, and management of laboratories had substantial reliability (\u03ba\u00a0=\u00a00.60, 0.62, and 0.62, respectively). The domains of positioning and needle insertion were less reliable (\u03ba\u00a0=\u00a00.16 and 0.16, respectively). Individuals who had completed one or more rotations in the neonatal intensive care unit (NICU<\/jats:styled-content>) had a higher total score (12.5 vs. 16.9; p\u00a0<\u00a00.01). The residents' own perception of ability to perform an LP<\/jats:styled-content> unsupervised did not result in a higher total score.<\/jats:p><\/jats:sec>Conclusions<\/jats:title>The OSATS<\/jats:styled-content>\u2010LP<\/jats:styled-content> has reasonable evidence in four of the five sources for test validity. This study serves as a launching point for using this tool in clinical environments such as the ED<\/jats:styled-content> and, therefore, has the potential to provide real\u2010time formative and summative feedback to improve resident skills and ultimately lead to improvements in patient care.<\/jats:p><\/jats:sec>","DOI":"10.1111\/acem.12093","type":"journal-article","created":{"date-parts":[[2013,3,20]],"date-time":"2013-03-20T17:02:58Z","timestamp":1363798978000},"page":"321-324","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Assessing the Validity Evidence of an Objective Structured Assessment Tool of Technical Skills for Neonatal Lumbar Punctures"],"prefix":"10.1111","volume":"20","author":[{"given":"Maya S.","family":"Iyer","sequence":"first","affiliation":[{"name":"Department of Pediatrics Division of Emergency Medicine Children's Hospital of Pittsburgh Pittsburgh PA"},{"name":"Department of Pediatrics and Communicable Diseases University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sally A.","family":"Santen","sequence":"additional","affiliation":[{"name":"Department of Medical Education University of Michigan Ann Arbor MI"},{"name":"Department of Emergency Medicine University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michele","family":"Nypaver","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kavita","family":"Warrier","sequence":"additional","affiliation":[{"name":"Department of Pediatrics and Communicable Diseases University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stuart","family":"Bradin","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rachel","family":"Chapman","sequence":"additional","affiliation":[{"name":"Division of Neonatal\u2010Perinatal Medicine University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jennifer","family":"McAllister","sequence":"additional","affiliation":[{"name":"Division of Neonatal\u2010Perinatal Medicine University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jennifer","family":"Vredeveld","sequence":"additional","affiliation":[{"name":"Department of Pediatrics and Communicable Diseases University of Michigan Ann Arbor MI"},{"name":"Department of Internal Medicine University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph B.","family":"House","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine University of Michigan Ann Arbor MI"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2013,3,20]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1553-2712.2001.tb00182.x"},{"key":"e_1_2_8_3_1","volume-title":"ACGME Program Requirements for Graduate Medical Education in Pediatrics","author":"Accreditation Council for Graduate Medical Education"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2005-0519"},{"key":"e_1_2_8_5_1","first-page":"1","article-title":"Blinded evaluation of interrater teliability of an operative competency assessment tool for direct laryngoscopy and rigid bronchoscopy","volume":"17","author":"Ishman SL","year":"2012","journal-title":"Arch Otolaryngol Head Neck Surg"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.4300\/JGME-D-11-00123.1"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2923.2009.03454.x"},{"key":"e_1_2_8_8_1","first-page":"1812","volume-title":"Textbook of Pediatric Emergency Medicine","author":"Carlson DW","year":"1999"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjmed.2005.10.036"},{"key":"e_1_2_8_11_1","doi-asserted-by":"crossref","DOI":"10.4324\/9780203880135","volume-title":"Assessment in Health Professions Education","author":"Downing SM","year":"2009"}],"container-title":["Academic Emergency Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Facem.12093","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/acem.12093","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,22]],"date-time":"2023-11-22T17:15:00Z","timestamp":1700673300000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/acem.12093"}},"subtitle":[],"editor":[{"given":"Daniel L.","family":"Theodoro","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2013,3]]},"references-count":10,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2013,3]]}},"alternative-id":["10.1111\/acem.12093"],"URL":"https:\/\/doi.org\/10.1111\/acem.12093","archive":["Portico"],"relation":{},"ISSN":["1069-6563","1553-2712"],"issn-type":[{"value":"1069-6563","type":"print"},{"value":"1553-2712","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,3]]},"assertion":[{"value":"2012-06-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-10-02","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-03-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}