{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T11:06:57Z","timestamp":1777892817463,"version":"3.51.4"},"reference-count":49,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2016,8,4]],"date-time":"2016-08-04T00:00:00Z","timestamp":1470268800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["J Learn Disabil"],"published-print":{"date-parts":[[2017,1]]},"abstract":"<jats:p>When classifying the evidence base of practices, special education scholars typically appraise study quality to identify and exclude from consideration in their reviews unacceptable-quality studies that are likely biased and might bias review findings if included. However, study quality appraisals used in the process of identifying evidence-based practices for students with learning and other disabilities have not been empirically validated (e.g., studies classified as unacceptable quality shown to have different, and presumably more biased, effects than high-quality studies). Using Gersten et al.\u2019s (2005) approach for appraising the quality of group experimental studies in special education, we examined whether (a) studies classified as unacceptable quality and high quality had meaningfully different effects and (b) unacceptable-quality studies were more likely to have outlying effects than high-quality studies among 36 group experimental studies that investigated the effectiveness of instructional practices for students with learning disabilities. Our preliminary analyses found that the effects of unacceptable-quality studies were not meaningfully different from the effects of high-quality studies. We discuss implications of these findings and call for more research to be conducted in this area.<\/jats:p>","DOI":"10.1177\/0022219415581178","type":"journal-article","created":{"date-parts":[[2015,4,17]],"date-time":"2015-04-17T21:52:19Z","timestamp":1429307539000},"page":"14-22","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":8,"title":["A Preliminary Investigation of the Empirical Validity of Study Quality Appraisal"],"prefix":"10.1177","volume":"50","author":[{"given":"Bryan G.","family":"Cook","sequence":"first","affiliation":[{"name":"University of Hawaii, Honolulu, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Danielle N.","family":"Dupuis","sequence":"additional","affiliation":[{"name":"University of Minnesota, Minneapolis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Asha K.","family":"Jitendra","sequence":"additional","affiliation":[{"name":"University of Minnesota, Minneapolis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2016,8,4]]},"reference":[{"key":"bibr1-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/001440290907500303"},{"key":"bibr2-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1001\/jama.287.22.2973"},{"key":"bibr3-0022219415581178","first-page":"221","volume-title":"The handbook of research synthesis and meta-analysis: Second edition","author":"Bornstein M.","year":"2009"},{"key":"bibr4-0022219415581178","doi-asserted-by":"publisher","DOI":"10.2106\/JBJS.F.01591"},{"key":"bibr5-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/001440290907500301"},{"key":"bibr6-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/00222194020350050101"},{"key":"bibr7-0022219415581178","volume-title":"Statistical power analysis for the behavioral sciences","author":"Cohen J.","year":"1988","edition":"2"},{"key":"bibr8-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/0741932514528995"},{"key":"bibr9-0022219415581178","first-page":"266","volume-title":"Enduring issues in special education: Personal perspectives","author":"Cook B. G.","year":"2015"},{"key":"bibr10-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/0741932514557271"},{"key":"bibr11-0022219415581178","unstructured":"Council for Exceptional Children. (2014). Council for Exceptional Children standards for evidence-based practices in special education. Retrieved from http:\/\/www.cec.sped.org\/~\/media\/Files\/Standards\/Evidence%20based%20Practices%20and%20Practice\/CEC%20EBP%20FINAL.pdf"},{"key":"bibr12-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2010.02.008"},{"key":"bibr13-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/019874290102700107"},{"key":"bibr14-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2004.03.017"},{"key":"bibr15-0022219415581178","doi-asserted-by":"publisher","DOI":"10.7861\/clinmedicine.1-6-478"},{"key":"bibr16-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/001440290507100202"},{"key":"bibr17-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwj069"},{"key":"bibr18-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/2.4.463"},{"key":"bibr19-0022219415581178","volume-title":"Visible learning: A synthesis of over 800 meta-analyses relating to achievement","author":"Hattie J.","year":"2013"},{"key":"bibr20-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2006.03.008"},{"key":"bibr21-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.d5928"},{"key":"bibr22-0022219415581178","unstructured":"Higgins J. P. T., Altman D. G., Sterne A. C. (2011). Assessing risk of bias in included studies. In Higgins J. P. T., Greene S. (Eds.), Cochrane handbook for systematic reviews of interventions (version 5.1.0). Retrieved from http:\/\/handbook.cochrane.org\/"},{"key":"bibr23-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/001440290507100203"},{"key":"bibr24-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0020124"},{"key":"bibr25-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1002\/jrsm.19"},{"key":"bibr26-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/001440291107700201"},{"key":"bibr27-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/0022219413487408"},{"key":"bibr28-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.323.7303.42"},{"key":"bibr29-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1001\/jama.282.11.1054"},{"key":"bibr30-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(99)00048-7"},{"key":"bibr31-0022219415581178","unstructured":"Lipsey M. W., Puzio K., Yun C., Hebert M. A., Steinka-Fry K., Cole M. W. \u2026 Busick M. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms (NCSER 2013-3000). Washington, DC: National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education. Retrieved from http:\/\/ies.ed.gov\/ncser\/pubs\/20133000\/pdf\/20133000.pdf"},{"key":"bibr32-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(98)01085-X"},{"key":"bibr33-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0040028"},{"key":"bibr34-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1080\/08989620500440287"},{"key":"bibr35-0022219415581178","unstructured":"National Autism Center. (2009). National standards report. Retrieved from http:\/\/www.nationalautismcenter.org\/pdf\/NAC%20Standards%20Report.pdf"},{"key":"bibr36-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/001440290507100201"},{"key":"bibr37-0022219415581178","doi-asserted-by":"publisher","DOI":"10.5694\/j.1326-5377.2006.tb00557.x"},{"key":"bibr38-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1016\/0021-9681(79)90012-2"},{"key":"bibr39-0022219415581178","first-page":"257","volume-title":"The handbook of research synthesis and meta-analysis","author":"Shadish W. R.","year":"2009","edition":"2"},{"key":"bibr40-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/002221940103400303"},{"key":"bibr41-0022219415581178","doi-asserted-by":"crossref","unstructured":"Soares H. P., Daniels S., Kumar A., Clarke M., Scott C., Swann S., Djulbegovic B. (2004). Bad reporting does not mean bad methods for randomised trials: Observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. British Medical Journal, 328(7430), 22\u201324. doi: http:\/\/dx.doi.org\/10.1136\/bmj.328.7430.22","DOI":"10.1136\/bmj.328.7430.22"},{"key":"bibr42-0022219415581178","volume-title":"Interventions for students with learning disabilities: A meta-analysis of treatment outcomes","author":"Swanson H. L.","year":"1999"},{"key":"bibr43-0022219415581178","doi-asserted-by":"publisher","DOI":"10.3102\/00346543068003277"},{"key":"bibr44-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1177\/07419325040250040801"},{"key":"bibr45-0022219415581178","first-page":"129","volume-title":"The handbook of research synthesis and meta-analysis","author":"Valentine J. C.","year":"2009","edition":"2"},{"key":"bibr46-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(00)00360-7"},{"key":"bibr47-0022219415581178","unstructured":"Viswanathan M., Ansari M. T., Berkman N. D., Chang S., Hartling L., McPheeters M. \u2026, Treadwell J. R. (2014). Assessing the risk of bias of individual studies in systematic reviews of health care interventions. In Methods guide for effectiveness and comparative effectiveness reviews (AHRQ Publication No. 10[14]-EHC063-EF, pp. 193\u2013214). Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from http:\/\/www.effectivehealthcare.ahrq.gov\/ehc\/products\/60\/318\/CER-Methods-guide-140109.pdf#page=198"},{"key":"bibr48-0022219415581178","unstructured":"What Works Clearinghouse. (2013). Procedures and standards handbook (version 3.0). Retrieved from http:\/\/ies.ed.gov\/ncee\/wwc\/pdf\/reference_resources\/wwc_procedures_v3_0_standards_handbook.pdf"},{"key":"bibr49-0022219415581178","doi-asserted-by":"publisher","DOI":"10.1353\/etc.2012.0012"}],"container-title":["Journal of Learning Disabilities"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0022219415581178","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0022219415581178","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0022219415581178","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T06:39:05Z","timestamp":1777617545000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0022219415581178"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,8,4]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,1]]}},"alternative-id":["10.1177\/0022219415581178"],"URL":"https:\/\/doi.org\/10.1177\/0022219415581178","relation":{},"ISSN":["0022-2194","1538-4780"],"issn-type":[{"value":"0022-2194","type":"print"},{"value":"1538-4780","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,8,4]]}}}