{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T19:52:14Z","timestamp":1792785134951,"version":"4.5.0"},"reference-count":36,"publisher":"American Speech Language Hearing Association","funder":[{"name":"AI Research Institutes program by National Science Foundation and the Institute of Education Sciences, U.S. Department of Education","award":["Award # 2229873 - National AI Institute for Exceptional Education"],"award-info":[{"award-number":["Award # 2229873 - National AI Institute for Exceptional Education"]}],"id":[{"id":"https:\/\/ror.org\/04et59085","id-type":"ROR","asserted-by":"crossref"}]},{"name":"National Institute on Deafness and Other Communication Disorders","award":["Grant R01DC011023"],"award-info":[{"award-number":["Grant R01DC011023"]}],"id":[{"id":"https:\/\/ror.org\/04mhx6838","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["LSHSS"],"abstract":"<jats:sec>\n                    <jats:title>Purpose:<\/jats:title>\n                    <jats:p>Artificial intelligence (AI) technologies have the potential to enhance the delivery of speech-language pathology services. One area of practice that could benefit from AI is automated screenings for developmental language disorder (DLD) in early elementary students. This report presents the elements and processes involved in evaluating an AI-based automated scoring system using archival recordings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method:<\/jats:title>\n                    <jats:p>A total of 947 audio samples of children completing the Redmond Sentence Recall (RSR) task, representing cases with and without DLD, were used. We developed an automated version of the RSR (AutoRSR), employing automated speech recognition (ASR), natural language processing, and machine learning models, to automate the tasks of transcribing recordings, tracking the number of errors children produced, scoring production accuracies based on RSR criteria, and generating pass\/fail decisions based on age-referenced expectations against a reference standard (i.e., Core Language scores from the Clinical Evaluation of Language Fundamentals\u2013Fourth Edition for a subset of 130 children). Different speech recognition models (Whisper and Reverb ASR) were applied to examine potential trade-offs. The accuracy of the AutoRSR was compared to human scoring.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>AutoRSR with Whisper speech recognition yielded sensitivity = .768 and specificity = .784, whereas Reverb speech recognition yielded sensitivity = .873 and specificity = .730. While AutoRSR had some mismatches with human evaluators and tended to produce lower scores, it converged more closely with the reference standard than human-scored protocols.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions:<\/jats:title>\n                    <jats:p>AI technologies have become powerful enough to automate the scoring of recordings of children's sentence recall with similar accuracy levels as human evaluators. Next steps are discussed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplemental Material:<\/jats:title>\n                    <jats:p>\n                      <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/doi.org\/10.23641\/asha.34050876\">https:\/\/doi.org\/10.23641\/asha.34050876<\/jats:ext-link>\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1044\/2026_lshss-25-00268","type":"journal-article","created":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T13:29:14Z","timestamp":1791552554000},"page":"1-12","source":"Crossref","is-referenced-by-count":0,"title":["Novel Approaches to Language Assessment Part 1: Artificial Intelligence\u2013Automated Scoring of a Sentence Recall Task for School-Age Children"],"prefix":"10.1044","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-6107-5612","authenticated-orcid":false,"given":"Dancheng","family":"Liu","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, University at Buffalo, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8458-4480","authenticated-orcid":false,"given":"Jason","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University at Buffalo, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2772-9531","authenticated-orcid":false,"given":"Carol A.","family":"Miller","sequence":"additional","affiliation":[{"name":"Department of Communication Sciences and Disorders, The Pennsylvania State University, University Park"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6814-8288","authenticated-orcid":false,"given":"Pamela A.","family":"Hadley","sequence":"additional","affiliation":[{"name":"Department of Speech and Hearing Science, University of Illinois Urbana\u2013Champaign"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3321-6022","authenticated-orcid":false,"given":"Sean M.","family":"Redmond","sequence":"additional","affiliation":[{"name":"Department of Communication Sciences and Disorders, The University of Utah, Salt Lake City"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2620-4859","authenticated-orcid":false,"given":"Jinjun","family":"Xiong","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University at Buffalo, NY"},{"name":"College of AI, Cyber and Computing, The University of Texas at San Antonio"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1310","published-online":{"date-parts":[[2026,10,8]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1044\/2022_JSLHR-21-00667"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.1044\/1092-4388(2009\/08-0099)"},{"key":"e_1_3_2_4_1","first-page":"470","article-title":"Automatic scoring of a sentence repetition task from voice recordings","volume":"9924","author":"Asgari M.","year":"2016","unstructured":"Asgari, M., Sliter, A., & Van Santen, J. (2016). Automatic scoring of a sentence repetition task from voice recordings. Text, Speech, and Dialogue, 9924, 470\u2013477. https:\/\/doi.org\/10.1007\/978-3-319-45510-5_54","journal-title":"Text, Speech, and Dialogue"},{"key":"e_1_3_2_5_1","doi-asserted-by":"crossref","unstructured":"Bain M. Huh J. Han T. & Zisserman A. (2023). WhisperX: Time-accurate speech transcription of long-form audio. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2303.00747","DOI":"10.21437\/Interspeech.2023-78"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1044\/2023_AJSLP-23-00286"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaac.2014.07.006"},{"key":"e_1_3_2_8_1","doi-asserted-by":"publisher","DOI":"10.1044\/2018_JSLHR-L-18-0036"},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.1044\/0161-1461(2012\/12-0093)"},{"key":"e_1_3_2_10_1","unstructured":"Bhandari N. Chen D. Fern\u00e1ndez M. \u00c1. D. R. Delworth N. Fox J. D. Jett\u00e9 M. McNamara Q. Miller C. Novotn\u00fd O. Profant J. Qin N. Ratajczak M. & Robichaud J. P. (2024). Reverb: Open-source ASR and diarization from Rev. arXiv. https:\/\/arxiv.org\/abs\/2410.03930"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1044\/2020_PERSP-19-00166"},{"key":"e_1_3_2_12_1","doi-asserted-by":"publisher","DOI":"10.1044\/2023_LSHSS-22-00169"},{"key":"e_1_3_2_13_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2022.894478"},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1044\/2021_JSLHR-21-00096"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC44109.2020.9175264"},{"key":"e_1_3_2_16_1","volume-title":"Developmental language disorder and social-emotional development: An introduction to theories, concepts, and research","author":"Gibson J.","year":"2024","unstructured":"Gibson, J., & Toseeb, U. (2024). Developmental language disorder and social-emotional development: An introduction to theories, concepts, and research. Oxford University Press. https:\/\/global.oup.com\/ukhe\/product\/developmental-language-disorder-and-social-emotional-development-9780192843845"},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.3390\/life14050557"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1044\/2025_JSLHR-24-00488"},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0305000914000130"},{"key":"e_1_3_2_20_1","unstructured":"Levenshtein V. I. (1966). Binary codes capable of correcting deletions insertions and reversals. Soviet Physics Doklady 10(8) 707\u2013710. https:\/\/nymity.ch\/sybilhunting\/pdf\/Levenshtein1966a.pdf [PDF]"},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.1093\/llc\/fqac089"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0305000924000345"},{"key":"e_1_3_2_23_1","unstructured":"Nassereldine A. Liu D. Xu C. Qin R. Shi Y. & Xiong J. (2024). PI-Whisper: designing an adaptive and incremental automatic speech recognition system for edge devices. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2406.15668"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/jcpp.12573"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1044\/2014_JSLHR-L-13-0189"},{"key":"e_1_3_2_26_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12126"},{"key":"e_1_3_2_27_1","doi-asserted-by":"crossref","unstructured":"Preza T. Liu D. Miller C. A Alabdallah A. Xiong J. Redmond S. M. & Hadley P. A. (2026). Novel approaches to language assessment Part 2: Evaluating measures of effort in a sentence recall task for school-age children. Language Speech and Hearing Services in Schools. Advance online publication. https:\/\/doi.org\/10.1044\/2026_LSHSS-25-00256","DOI":"10.1044\/2026_LSHSS-25-00256"},{"key":"e_1_3_2_28_1","first-page":"28492","volume-title":"Proceedings of the 40th International Conference on Machine Learning","author":"Radford A.","year":"2023","unstructured":"Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2023, July). Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning (Vol. 202, pp. 28492\u201328518). PMLR. https:\/\/proceedings.mlr.press\/v202\/radford23a.html"},{"key":"e_1_3_2_29_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2024-098290"},{"key":"e_1_3_2_30_1","unstructured":"Redmond S. (2019). Redmond Sentence Recall. Retrieved December 30 2024 from https:\/\/health.utah.edu\/sites\/g\/files\/zrelqx131\/files\/files\/migration\/image\/rsr-manual.pdf [PDF]"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1044\/2019_JSLHR-L-18-0388"},{"key":"e_1_3_2_32_1","volume-title":"Clinical Evaluation of Language Fundamentals\u2013Fourth Edition (CELF-4)","author":"Semel E.","year":"2003","unstructured":"Semel, E., Wiig, E. H., & Secord, W. A. (2003). Clinical Evaluation of Language Fundamentals\u2013Fourth Edition (CELF-4). The Psychological Corporation. https:\/\/www.pearsonassessments.com\/en-us\/Store\/Professional-Assessments\/Speech-%26-Language\/Clinical-Evaluation-of-Language-Fundamentals-Screening-Test-%7C-Fourth-Edition\/p\/100000438"},{"key":"e_1_3_2_33_1","doi-asserted-by":"publisher","DOI":"10.1044\/2025_LSHSS-25-00008"},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/SCEECS48394.2020.190"},{"key":"e_1_3_2_35_1","doi-asserted-by":"publisher","DOI":"10.1044\/2024_JSLHR-23-00490"},{"key":"e_1_3_2_36_1","doi-asserted-by":"publisher","DOI":"10.1002\/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3"},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1044\/2024_JSLHR-24-00157"}],"container-title":["Language, Speech, and Hearing Services in Schools"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.asha.org\/doi\/pdf\/10.1044\/2026_LSHSS-25-00268","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T18:33:45Z","timestamp":1791830025000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.asha.org\/doi\/10.1044\/2026_LSHSS-25-00268"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,8]]},"references-count":36,"alternative-id":["10.1044\/2026_LSHSS-25-00268"],"URL":"https:\/\/doi.org\/10.1044\/2026_lshss-25-00268","relation":{},"ISSN":["0161-1461","1558-9129"],"issn-type":[{"value":"0161-1461","type":"print"},{"value":"1558-9129","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,8]]}}}