{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T21:02:46Z","timestamp":1793134966526,"version":"4.5.0"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T00:00:00Z","timestamp":1736812800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T00:00:00Z","timestamp":1736812800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Behav Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    The Rapid Online Assessment of Reading (ROAR) is a web-based lexical decision task that measures single-word reading abilities in children and adults without a proctor. Here we study whether item response theory (IRT) and computerized adaptive testing (CAT) can be used to create a more efficient online measure of word recognition. To construct an item bank, we first analyzed data taken from four groups of students (\n                    <jats:italic>N<\/jats:italic>\n                    \u2009=\u20091960) who differed in age, socioeconomic status, and language-based learning disabilities. The majority of item parameters were highly consistent across groups (\n                    <jats:italic>r<\/jats:italic>\n                    \u2009=\u2009.78\u2013.94), and six items that functioned differently across groups were removed. Next, we implemented a JavaScript CAT algorithm and conducted a validation experiment with 485 students in grades 1\u20138 who were randomly assigned to complete trials of all items in the item bank in either (a) a random order or (b) a CAT order. We found that, to achieve reliability of 0.9, CAT improved test efficiency by 40%: 75 CAT items produced the same standard error of measurement as 125 items in a random order. Subsequent validation in 32 public school classrooms showed that an approximately 3-min ROAR-CAT can achieve high correlations (\n                    <jats:italic>r<\/jats:italic>\n                    \u2009=\u2009.89 for first grade,\n                    <jats:italic>r<\/jats:italic>\n                    \u2009=\u2009.73 for second grade) with alternative 5\u201315-min individually proctored oral reading assessments. Our findings suggest that ROAR-CAT is a promising tool for efficiently and accurately measuring single-word reading ability. Furthermore, our development process serves as a model for creating adaptive online assessments that bridge research and practice.\n                  <\/jats:p>","DOI":"10.3758\/s13428-024-02578-y","type":"journal-article","created":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T12:07:52Z","timestamp":1736856472000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["ROAR-CAT: Rapid Online Assessment of Reading ability with Computerized Adaptive Testing"],"prefix":"10.3758","volume":"57","author":[{"given":"Wanjing Anya","family":"Ma","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam","family":"Richie-Halford","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amy K.","family":"Burkhardt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klint","family":"Kanopka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Clementine","family":"Chou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin W.","family":"Domingue","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jason D.","family":"Yeatman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,14]]},"reference":[{"issue":"1","key":"2578_CR1","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1002\/dys.1461","volume":"20","author":"S Ara\u00fajo","year":"2014","unstructured":"Ara\u00fajo, S., Fa\u00edsca, L., Bram\u00e3o, I., Petersson, K. M., & Reis, A. (2014). Lexical and phonological processes in dyslexic readers: Evidence from a visual lexical decision task. Dyslexia, 20(1), 38\u201353.","journal-title":"Dyslexia"},{"key":"2578_CR2","doi-asserted-by":"crossref","unstructured":"Barrada, J. R., Olea, J., Ponsoda, V., & Abad, F. J. (2008). Incorporating randomness in the Fisher information for improving item-exposure control in CATs. The British journal of mathematical and statistical psychology, 61(Pt 2), 493\u2013513. Wiley.","DOI":"10.1348\/000711007X230937"},{"key":"2578_CR3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v048.i06","volume":"48","author":"PR Chalmers","year":"2012","unstructured":"Chalmers, P. R. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48, 1\u201329.","journal-title":"Journal of Statistical Software"},{"issue":"1","key":"2578_CR4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11336-014-9401-5","volume":"80","author":"H-H Chang","year":"2015","unstructured":"Chang, H.-H. (2015). Psychometrics behind Computerized Adaptive Testing. Psychometrika, 80(1), 1\u201320.","journal-title":"Psychometrika"},{"issue":"3","key":"2578_CR5","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1177\/014662169602000303","volume":"20","author":"HH Chang","year":"1996","unstructured":"Chang, H. H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213\u2013229.","journal-title":"Applied Psychological Measurement"},{"key":"2578_CR6","doi-asserted-by":"crossref","unstructured":"Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74(4), 619\u2013632. Springer Science and Business Media LLC.","DOI":"10.1007\/s11336-009-9123-2"},{"issue":"6","key":"2578_CR7","first-page":"1","volume":"70","author":"SW Choi","year":"2010","unstructured":"Choi, S. W., Grady, M. W., & Dodd, B. G. (2010). A New Stopping Rule for Computerized Adaptive Testing. Educational and Psychological Measurement, 70(6), 1\u201317.","journal-title":"Educational and Psychological Measurement"},{"key":"2578_CR8","unstructured":"Christ, T. J., Ara\u00f1as, Y. A., Johnson, L., Kember, J. M., Kilgus, S., Kiss, A., McCarthy Trentman, A. M., et al. (2018). Formative assessment system for teachers technical manual. Minneapolis, MN: Author and FastBridge Learning."},{"issue":"1","key":"2578_CR9","doi-asserted-by":"crossref","first-page":"48","DOI":"10.3102\/0013189X16631750","volume":"45","author":"CE Coburn","year":"2016","unstructured":"Coburn, C. E., & Penuel, W. R. (2016). Research\u2013practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48\u201354.","journal-title":"Educational Researcher"},{"key":"2578_CR10","volume-title":"Fountas and Pinnell Leveled Literacy Intervention (LLI) Orange (Grade K) Program Guide","author":"IC Fountas","year":"2009","unstructured":"Fountas, I. C., & Pinnell, G. S. (2009). Fountas and Pinnell Leveled Literacy Intervention (LLI) Orange (Grade K) Program Guide. Heinemann."},{"key":"2578_CR11","volume-title":"Experimentology: An open science approach to experimental psychology methods","author":"MC Frank","year":"2023","unstructured":"Frank, M. C., Braginsky, M., Cachia, J., Coles, N., Hardwicke, T., Hawkins, R., Mathur, M. B., et al. (2023). Experimentology: An open science approach to experimental psychology methods. MIT Press."},{"issue":"1","key":"2578_CR12","doi-asserted-by":"crossref","first-page":"10249","DOI":"10.1038\/s41598-024-60834-9","volume":"14","author":"L Gijbels","year":"2024","unstructured":"Gijbels, L., Burkhardt, A., Ma, W. A., & Yeatman, J. D. (2024). Rapid online assessment of reading and phonological awareness (ROAR-PA). Scientific Reports, 14(1), 10249.","journal-title":"Scientific Reports"},{"key":"2578_CR13","doi-asserted-by":"crossref","unstructured":"Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical Guidelines for Assessing Computerized Adaptive Tests. Journal of Educational Measurement, 21(4), 347\u2013360. [National Council on Measurement in Education, Wiley].","DOI":"10.1111\/j.1745-3984.1984.tb01039.x"},{"issue":"4","key":"2578_CR14","doi-asserted-by":"crossref","first-page":"708","DOI":"10.1037\/met0000152","volume":"23","author":"TD Jorgensen","year":"2018","unstructured":"Jorgensen, T. D., Kite, B. A., Chen, P.-Y., & Short, S. D. (2018). Permutation randomization methods for testing measurement equivalence and detecting differential item functioning in multiple-group confirmatory factor analysis. Psychological Methods, 23(4), 708\u2013728.","journal-title":"Psychological Methods"},{"issue":"6","key":"2578_CR15","doi-asserted-by":"crossref","first-page":"2288","DOI":"10.1044\/2022_JSLHR-21-00372","volume":"65","author":"G Kachergis","year":"2022","unstructured":"Kachergis, G., Marchman, V. A., Dale, P. S., Mankewitz, J., & Frank, M. C. (2022). Online computerized adaptive tests of children\u2019s vocabulary development in English and Mexican Spanish. Journal of Speech, Language, and Hearing Research, 65(6), 2288\u20132308.","journal-title":"Journal of Speech, Language, and Hearing Research"},{"key":"2578_CR16","doi-asserted-by":"publisher","unstructured":"Kanopka, K., & Domingue, B. W. (2024). A position-sensitive mixture item response model. Journal of Educational and Behavioral Statistics. https:\/\/doi.org\/10.3102\/10769986241289399","DOI":"10.3102\/10769986241289399"},{"issue":"1","key":"2578_CR17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3758\/s13428-014-0458-y","volume":"47","author":"JR de Leeuw","year":"2015","unstructured":"de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 47(1), 1\u201312.","journal-title":"Behavior Research Methods"},{"key":"2578_CR18","doi-asserted-by":"crossref","unstructured":"Lord, F. M. (1971). THE SELF-SCORING FLEXILEVEL TEST 1. Journal of Educational Measurement, 8(3), 147\u2013151. Wiley Online Library.","DOI":"10.1111\/j.1745-3984.1971.tb00918.x"},{"key":"2578_CR19","doi-asserted-by":"crossref","DOI":"10.4324\/9780203056615","volume-title":"Applications of item response theory to practical testing problems","author":"FM Lord","year":"2012","unstructured":"Lord, F. M. (2012). Applications of item response theory to practical testing problems. Routledge."},{"key":"2578_CR20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v076.c01","volume":"76","author":"D Magis","year":"2017","unstructured":"Magis, D., & Barrada, J. R. (2017). Computerized Adaptive Testing with R: Recent Updates of the Package catR. Journal of Statistical Software, 76, 1\u201319.","journal-title":"Journal of Statistical Software"},{"issue":"2","key":"2578_CR21","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.bandl.2006.04.003","volume":"98","author":"VEG Martens","year":"2006","unstructured":"Martens, V. E. G., & de Jong, P. F. (2006). The effect of word length on lexical decision in dyslexic and normal reading children. Brain and Language, 98(2), 140\u2013149.","journal-title":"Brain and Language"},{"issue":"3","key":"2578_CR22","doi-asserted-by":"crossref","DOI":"10.1111\/desc.13458","volume":"27","author":"M Ramamurthy","year":"2024","unstructured":"Ramamurthy, M., White, A. L., & Yeatman, J. D. (2024). Children with dyslexia show no deficit in exogenous spatial attention but show differences in visual encoding. Developmental Science, 27(3), e13458.","journal-title":"Developmental Science"},{"issue":"1","key":"2578_CR23","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1037\/0033-295X.111.1.159","volume":"111","author":"R Ratcliff","year":"2004","unstructured":"Ratcliff, R., Gomez, P., & McKoon, G. (2004). A diffusion model account of the lexical decision task. Psychological Review, 111(1), 159\u2013182.","journal-title":"Psychological Review"},{"key":"2578_CR24","doi-asserted-by":"crossref","unstructured":"Rupp, A. A., & Zumbo, B. D. (2004). A Note on How to Quantify and Report Whether Irt Parameter Invariance Holds: When Pearson Correlations are Not Enough. Educational and psychological measurement, 64(4), 588\u2013599. SAGE Publications Inc.","DOI":"10.1177\/0013164403261051"},{"key":"2578_CR25","doi-asserted-by":"crossref","unstructured":"Rupp, A. A., & Zumbo, B. D. (2006). Understanding Parameter Invariance in Unidimensional IRT Models. Educational and psychological measurement, 66(1), 63\u201384. SAGE Publications Inc.","DOI":"10.1177\/0013164404273942"},{"key":"2578_CR26","doi-asserted-by":"crossref","unstructured":"Sabatini, J., Weeks, J., O\u2019Reilly, T., Bruce, K., Steinberg, J., & Chao, S.-F. (2019). SARA reading components tests, RISE forms: Technical adequacy and test design, 3rd edition. ETS research report series, 2019(1), 1\u201330. Wiley.","DOI":"10.1002\/ets2.12269"},{"key":"2578_CR27","doi-asserted-by":"crossref","unstructured":"Stocking, M. L. (1988). Scale drift in on\u2010line calibration. ETS research report series, 1988(1), i\u2013122. Wiley.","DOI":"10.1002\/j.2330-8516.1988.tb00284.x"},{"issue":"1","key":"2578_CR28","first-page":"1","volume":"16","author":"NA Thompson","year":"2011","unstructured":"Thompson, N. A., & Weiss, D. A. (2011). A framework for the development of computerized adaptive tests. Practical Assessment, Research, and Evaluation, 16(1), 1.","journal-title":"Practical Assessment, Research, and Evaluation"},{"key":"2578_CR29","doi-asserted-by":"crossref","unstructured":"Van der Linden, W. J., & Glas, C. A. W. (2000). Computerized adaptive testing: Theory and practice. Springer.","DOI":"10.1007\/0-306-47531-6"},{"key":"2578_CR30","doi-asserted-by":"crossref","unstructured":"Van der Linden, W. J., & Glas, C. A. W. (2010). Elements of adaptive testing (Vol. 10). Springer.","DOI":"10.1007\/978-0-387-85461-8"},{"issue":"1","key":"2578_CR31","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1111\/j.1745-3984.2001.tb01115.x","volume":"38","author":"T Wang","year":"2001","unstructured":"Wang, T., & Kolen, M. J. (2001). Evaluating comparability in computerized adaptive testing: Issues, criteria and an example. Journal of Educational Measurement, 38(1), 19\u201349.","journal-title":"Journal of Educational Measurement"},{"key":"2578_CR32","doi-asserted-by":"crossref","unstructured":"Wang, Z., Sabatini, J., & O\u2019Reilly, T. (2020). When Slower is Faster: Time Spent Decoding Novel Words Predicts Better Decoding and Faster Growth. Scientific studies of reading: the official journal of the Society for the Scientific Study of Reading, 24(5), 397\u2013410. Routledge.","DOI":"10.1080\/10888438.2019.1696347"},{"key":"2578_CR33","doi-asserted-by":"crossref","unstructured":"Weiss, D. J. (1982). Improving Measurement Quality and Efficiency with Adaptive Testing. Applied psychological measurement, 6(4), 473\u2013492. SAGE Publications Inc.","DOI":"10.1177\/014662168200600408"},{"key":"2578_CR34","doi-asserted-by":"crossref","unstructured":"Weiss, D. J., & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of educational measurement, 21(4), 361\u2013375. Wiley.","DOI":"10.1111\/j.1745-3984.1984.tb01040.x"},{"key":"2578_CR35","doi-asserted-by":"crossref","unstructured":"Wentworth, L., Arce-Trigatti, P., Conaway, C., & Shewchuk, S. (2023). Brokering in Education Research-Practice Partnerships: A Guide for Education Professionals and Researchers. Taylor & Francis.","DOI":"10.4324\/9781003334385"},{"key":"2578_CR36","doi-asserted-by":"crossref","unstructured":"Wilcox, R. R. (2011). Introduction to robust estimation and hypothesis testing. Academic press.","DOI":"10.1016\/B978-0-12-386983-8.00001-9"},{"issue":"1","key":"2578_CR37","doi-asserted-by":"crossref","first-page":"6396","DOI":"10.1038\/s41598-021-85907-x","volume":"11","author":"JD Yeatman","year":"2021","unstructured":"Yeatman, J. D., Tang, K. A., Donnelly, P. M., Yablonski, M., Ramamurthy, M., Karipidis, I. I., Caffarra, S., et al. (2021). Rapid online assessment of reading ability. Scientific Reports, 11(1), 6396.","journal-title":"Scientific Reports"},{"key":"2578_CR38","doi-asserted-by":"crossref","unstructured":"Yeatman, J. D., Tran, J. E., Burkhardt, A. K., Ma, W. A., Mitchell, J., Yablonski, M., Gijbels, L., et al. (2024). Development and validation of a rapid and precise online sentence reading efficiency assessment. Frontiers in education, 9, 1494431. Frontiers.","DOI":"10.3389\/feduc.2024.1494431"}],"container-title":["Behavior Research Methods"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.3758\/s13428-024-02578-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.3758\/s13428-024-02578-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.3758\/s13428-024-02578-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T21:43:39Z","timestamp":1752702219000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.3758\/s13428-024-02578-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,14]]},"references-count":38,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1]]}},"alternative-id":["2578"],"URL":"https:\/\/doi.org\/10.3758\/s13428-024-02578-y","relation":{},"ISSN":["1554-3528"],"issn-type":[{"value":"1554-3528","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,14]]},"assertion":[{"value":"4 December 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 January 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and\/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved by the Stanford Institutional Review Board (IRB).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"All individual participants or their parents\/guardians were provided with consent information and were given ample opportunity to opt out of participation in the studies.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Data available on request for research collaboration purposes, and none of the experiments was preregistered.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Open practices statement"}},{"value":"The authors declare that they have no conflicts of interest or competing interests.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest\/Competing interests"}}],"article-number":"56"}}