{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:29:05Z","timestamp":1750307345297,"version":"3.41.0"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2011,8,1]],"date-time":"2011-08-01T00:00:00Z","timestamp":1312156800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Speech Lang. Process."],"published-print":{"date-parts":[[2011,8]]},"abstract":"<jats:p>Automatic literacy assessment is an area of research that has shown significant progress in recent years. Technology can be used to automatically administer reading tasks and analyze and interpret children's reading skills. It has the potential to transform the classroom dynamic by providing useful information to teachers in a repeatable, consistent, and affordable way. While most previous research has focused on automatically assessing children reading words and sentences, assessments of children's earlier foundational skills is needed. We address this problem in this research by automatically verifying preliterate children's pronunciations of English letter-names and the sounds each letter represents (\u201cletter-sounds\u201d). The children analyzed in this study were from a diverse bilingual background and were recorded in actual kindergarten to second grade classrooms. We first manually verified (accept\/reject) the letter-name and letter-sound utterances, which serve as the ground-truth in this study. Next, we investigated four automatic verification methods that were based on automatic speech recognition techniques. We attained percent agreement with human evaluations of 90% and 85% for the letter-name and letter-sound tasks, respectively. Humans agree between themselves an average of 95% of the time for both tasks. We discuss the various confounding factors for this assessment task, such as background noise and the presence of disfluencies, that impact automatic verification performance.<\/jats:p>","DOI":"10.1145\/1998384.1998389","type":"journal-article","created":{"date-parts":[[2011,8,16]],"date-time":"2011-08-16T19:11:58Z","timestamp":1313521918000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Automatically assessing the ABCs"],"prefix":"10.1145","volume":"7","author":[{"given":"Matthew P.","family":"Black","sequence":"first","affiliation":[{"name":"Signal Analysis &amp; Interpretation Laboratory, University of Southern California, Los Angeles, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abe","family":"Kazemzadeh","sequence":"additional","affiliation":[{"name":"Signal Analysis &amp; Interpretation Laboratory, University of Southern California, Los Angeles, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph","family":"Tepperman","sequence":"additional","affiliation":[{"name":"Rosetta Stone Labs, Boulder, CO"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shrikanth S.","family":"Narayanan","sequence":"additional","affiliation":[{"name":"Signal Analysis &amp; Interpretation Laboratory, University of Southern California, Los Angeles, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2011,8,18]]},"reference":[{"volume-title":"Proceedings of the International Workshop on Multimedia Signal Processing (MMSP).","author":"Alwan A.","key":"e_1_2_1_1_1","unstructured":"Alwan , A. , Bai , Y. , Black , M. P. , Casey , L. , Gerosa , M. , Heritage , M. , Iseli , M. , Jones , B. , Kazemzadeh , A. , Lee , S. , Narayanan , S. , Price , P. , Tepperman , J. , and Wang , S . 2007. A system for technology based assessment of language and literacy in young children: The role of multiple information sources . In Proceedings of the International Workshop on Multimedia Signal Processing (MMSP). Alwan, A., Bai, Y., Black, M. P., Casey, L., Gerosa, M., Heritage, M., Iseli, M., Jones, B., Kazemzadeh, A., Lee, S., Narayanan, S., Price, P., Tepperman, J., and Wang, S. 2007. A system for technology based assessment of language and literacy in young children: The role of multiple information sources. In Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01027275"},{"volume-title":"Proceedings of Interspeech.","author":"Black M. P.","key":"e_1_2_1_3_1","unstructured":"Black , M. P. , Tepperman , J. , Kazemzadeh , A. , Lee , S. , and Narayanan , S . 2008. Pronunciation verification of English letter-sounds in preliterate children . In Proceedings of Interspeech. Black, M. P., Tepperman, J., Kazemzadeh, A., Lee, S., and Narayanan, S. 2008. Pronunciation verification of English letter-sounds in preliterate children. In Proceedings of Interspeech."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2009.4960720"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2010.2076389"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1080\/0969595980050102"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/PL00000018"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csl.2008.03.001"},{"volume-title":"Proceedings of the InSTIL\/ICALL Symposium on Computer Assisted Learning.","author":"Cosi P.","key":"e_1_2_1_9_1","unstructured":"Cosi , P. , Delmonte , R. , Biscetti , S. , Cole , R. A. , Pellom , B. , and Vuren , S . 2004. Italian literacy tutor-tools and technologies for individuals with cognitive disabilities . In Proceedings of the InSTIL\/ICALL Symposium on Computer Assisted Learning. Cosi, P., Delmonte, R., Biscetti, S., Cole, R. A., Pellom, B., and Vuren, S. 2004. Italian literacy tutor-tools and technologies for individuals with cognitive disabilities. In Proceedings of the InSTIL\/ICALL Symposium on Computer Assisted Learning."},{"volume-title":"Proceedings of Interspeech.","author":"Duchateau J.","key":"e_1_2_1_10_1","unstructured":"Duchateau , J. , Cleuren , L. , Van Hamme , H. , and Ghesqui\u00e8re , P . 2007. Automatic assessment of children's reading level . In Proceedings of Interspeech. Duchateau, J., Cleuren, L., Van Hamme, H., and Ghesqui\u00e8re, P. 2007. Automatic assessment of children's reading level. In Proceedings of Interspeech."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2009.04.005"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2007.05.004"},{"volume-title":"Formative assessment: What do teachers need to know and do&quest","author":"Heritage M.","key":"e_1_2_1_13_1","unstructured":"Heritage , M. 2007. Formative assessment: What do teachers need to know and do&quest ; Phi Delta Kappan 89, 2, 140--145. Heritage, M. 2007. Formative assessment: What do teachers need to know and do&quest; Phi Delta Kappan 89, 2, 140--145."},{"volume-title":"Proceedings of Interspeech.","author":"Kazemzadeh A.","key":"e_1_2_1_14_1","unstructured":"Kazemzadeh , A. , Tepperman , J. , Silva , J. , You , H. , Lee , S. , Alwan , A. , and Narayanan , S . 2006. Automatic detection of voice onset time contrasts for use in pronunciation assessment . In Proceedings of Interspeech. Kazemzadeh, A., Tepperman, J., Silva, J., You, H., Lee, S., Alwan, A., and Narayanan, S. 2006. Automatic detection of voice onset time contrasts for use in pronunciation assessment. In Proceedings of Interspeech."},{"volume-title":"Proceedings of Interspeech.","author":"Kazemzadeh A.","key":"e_1_2_1_15_1","unstructured":"Kazemzadeh , A. , You , H. , Iseli , M. , Jones , B. , Cui , X. , Heritage , M. , Price , P. , Anderson , E. , Narayanan , S. , and Alwan , A . 2005. TBALL data collection: The making of a young children's speech corpus . In Proceedings of Interspeech. Kazemzadeh, A., You, H., Iseli, M., Jones, B., Cui, X., Heritage, M., Price, P., Anderson, E., Narayanan, S., and Alwan, A. 2005. TBALL data collection: The making of a young children's speech corpus. In Proceedings of Interspeech."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.426686"},{"key":"e_1_2_1_17_1","first-page":"285","article-title":"The ABCs of the ABCs: The development of letter-name and letter-sound knowledge","volume":"45","author":"McBride-Chang C.","year":"1999","unstructured":"McBride-Chang , C. 1999 . The ABCs of the ABCs: The development of letter-name and letter-sound knowledge . Merrill-Palmer Quart. 45 , 2, 285 -- 308 . McBride-Chang, C. 1999. The ABCs of the ABCs: The development of letter-name and letter-sound knowledge. Merrill-Palmer Quart. 45, 2, 285--308.","journal-title":"Merrill-Palmer Quart."},{"volume-title":"Proceedings of AAAI.","author":"Mostow J.","key":"e_1_2_1_18_1","unstructured":"Mostow , J. , Roth , S. F. , Hauptmann , E. G. , and Kane , M . 1994. A prototype reading coach that listens . In Proceedings of AAAI. Mostow, J., Roth, S. F., Hauptmann, E. G., and Kane, M. 1994. A prototype reading coach that listens. In Proceedings of AAAI."},{"volume-title":"Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Tech. rep. 00-4769","author":"National Reading Panel. 2000.","key":"e_1_2_1_19_1","unstructured":"National Reading Panel. 2000. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Tech. rep. 00-4769 , National Institute for Child Health and Human Development , National Institute of Health, Washington, D.C. National Reading Panel. 2000. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Tech. rep. 00-4769, National Institute for Child Health and Human Development, National Institute of Health, Washington, D.C."},{"volume-title":"Classroom Literacy Assessment: Making Sense of What Students Know and Do","author":"Paratore J.","key":"e_1_2_1_20_1","unstructured":"Paratore , J. and McCormack , R. 2007. Classroom Literacy Assessment: Making Sense of What Students Know and Do . Guilford Press , New York, NY . Paratore, J. and McCormack, R. 2007. Classroom Literacy Assessment: Making Sense of What Students Know and Do. Guilford Press, New York, NY."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2009.05.001"},{"volume-title":"Proceedings of Interspeech.","author":"Rasmussen M. H.","key":"e_1_2_1_22_1","unstructured":"Rasmussen , M. H. , Tan , Z. H. , Lindberg , B. , and Jensen , S. H . 2009. A system for detecting miscues in dyslexic read speech . In Proceedings of Interspeech. Rasmussen, M. H., Tan, Z. H., Lindberg, B., and Jensen, S. H. 2009. A system for detecting miscues in dyslexic read speech. In Proceedings of Interspeech."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2010.2047812"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/2907324.2907378"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-6393(99)00044-8"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csl.2009.12.004"},{"volume-title":"Proceedings of Interspeech.","author":"You H.","key":"e_1_2_1_27_1","unstructured":"You , H. , Alwan , A. , Kazemzadeh , A. , and Narayanan , S . 2005. Pronunciation variations of Spanish-accented English spoken by young children . In Proceedings of Interspeech. You, H., Alwan, A., Kazemzadeh, A., and Narayanan, S. 2005. Pronunciation variations of Spanish-accented English spoken by young children. In Proceedings of Interspeech."},{"key":"e_1_2_1_28_1","unstructured":"Young S. Evermann G. Gales M. Hain T. Kershaw D. Liu X. Moore G. Odell J. Ollason D. Povey D. Valtchev V. and Woodland P. C. 2006. The HTK Book (version 3.4). Cambridge University Engineering Department.  Young S. Evermann G. Gales M. Hain T. Kershaw D. Liu X. Moore G. Odell J. Ollason D. Povey D. Valtchev V. and Woodland P. C. 2006. The HTK Book (version 3.4). Cambridge University Engineering Department."},{"volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP).","author":"Zheng J.","key":"e_1_2_1_29_1","unstructured":"Zheng , J. , Huang , C. , Chu , M. , Soong , F. K. , and Ye , W . 2007. Generalized segment posterior probability for automatic Mandarin pronunciation evaluation . In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Zheng, J., Huang, C., Chu, M., Soong, F. K., and Ye, W. 2007. Generalized segment posterior probability for automatic Mandarin pronunciation evaluation. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)."}],"container-title":["ACM Transactions on Speech and Language Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1998384.1998389","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1998384.1998389","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T11:06:27Z","timestamp":1750244787000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1998384.1998389"}},"subtitle":["Verification of children's spoken letter-names and letter-sounds"],"short-title":[],"issued":{"date-parts":[[2011,8]]},"references-count":29,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2011,8]]}},"alternative-id":["10.1145\/1998384.1998389"],"URL":"https:\/\/doi.org\/10.1145\/1998384.1998389","relation":{},"ISSN":["1550-4875","1550-4883"],"issn-type":[{"type":"print","value":"1550-4875"},{"type":"electronic","value":"1550-4883"}],"subject":[],"published":{"date-parts":[[2011,8]]},"assertion":[{"value":"2010-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-08-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}