{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T04:34:23Z","timestamp":1776659663612,"version":"3.51.2"},"reference-count":57,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T00:00:00Z","timestamp":1723420800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Res. Metr. Anal."],"abstract":"<jats:p>An increasingly popular form of collaboration involves forming partnerships among researchers, educators, and community members to improve or transform education systems through research inquiry. However, not all partnerships are successful. The field needs valid, reliable, and useful measures to help with assessing progress toward partnership goals. In this community case study, we present a participatory, mixed-methods approach for creating measures to assess the progress of education research-practice partnerships (RPPs). The case illustrates a novel approach to measurement design, driven by perspectives and feedback of over 300 members of 80 partnerships. As a result, the measures align with the values and practices of the very collaborations the measures were intended to assess.<\/jats:p>","DOI":"10.3389\/frma.2024.1210547","type":"journal-article","created":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T07:04:44Z","timestamp":1723446284000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Designing measures of complex collaborations with participatory, evidence-centered design"],"prefix":"10.3389","volume":"9","author":[{"given":"Caitlin C.","family":"Farrell","sequence":"first","affiliation":[]},{"given":"William R.","family":"Penuel","sequence":"additional","affiliation":[]},{"given":"Paula","family":"Arce-Trigatti","sequence":"additional","affiliation":[]},{"given":"James","family":"Soland","sequence":"additional","affiliation":[]},{"given":"Corinne","family":"Singleton","sequence":"additional","affiliation":[]},{"given":"Alison Fox","family":"Resnick","sequence":"additional","affiliation":[]},{"given":"Kristina","family":"Stamatis","sequence":"additional","affiliation":[]},{"given":"Robbin","family":"Riedy","sequence":"additional","affiliation":[]},{"given":"Erin","family":"Henrick","sequence":"additional","affiliation":[]},{"given":"Stacey","family":"Sexton","sequence":"additional","affiliation":[]},{"given":"Sarah","family":"Wellberg","sequence":"additional","affiliation":[]},{"given":"Danny","family":"Schmidt","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,8,12]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"70","DOI":"10.18608\/jla.2019.62.5","article-title":"Designing in context: reaching beyond usability in learning analytics dashboard design","volume":"6","author":"Ahn","year":"2019","journal-title":"J. Learn. Analyt"},{"key":"B2","year":"2014","journal-title":"Standards for Educational and Psychological Testing"},{"key":"B3","doi-asserted-by":"publisher","first-page":"853","DOI":"10.3389\/fpsyg.2019.00853","article-title":"The expanded evidence-centered design (e-ECD) for learning and assessment systems: a framework for incorporating learning goals and processes within assessment design","volume":"10","author":"Arieli-Attali","year":"2019","journal-title":"Front. Psychol."},{"key":"B4","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1016\/0149-7189(87)90038-3","article-title":"Stakeholders as partners in evaluation: a stakeholder-collaborative approach","volume":"10","author":"Ayers","year":"1987","journal-title":"Eval. Progr. Plan"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1811","DOI":"10.1002\/jcop.22372","article-title":"What sets the conditions for success in community-partnered evaluation work? Multiple perspectives on a small-scale research-practice partnership evaluation","volume":"48","author":"Ballard","year":"2020","journal-title":"J. Commun. Psychol"},{"key":"B6","doi-asserted-by":"publisher","first-page":"337","DOI":"10.18865\/ed.27.S1.337","article-title":"The value of social network analysis for evaluating academic-community partnerships and collaborations for social determinants of health research","volume":"27","author":"Bright","year":"2017","journal-title":"Ethnicity Dis."},{"key":"B7","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1016\/S0149-7189(02)00017-4","article-title":"Assessing and improving partnership relationships and outcomes: a proposed framework","volume":"25","author":"Brinkerhoff","year":"2002","journal-title":"Eval. Progr. Plan"},{"key":"B8","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1177\/1098214018804195","article-title":"Assessing triangulation across methodologies, methods, and stakeholder groups: the joys, woes, and politics of interpreting convergent and divergent data","volume":"41","author":"Campbell","year":"2020","journal-title":"Am. J. Evol"},{"key":"B9","volume-title":"Research-Practice Partnerships at the District Level: A New Strategy for Leveraging Research for Educational Improvement.","author":"Coburn","year":"2013"},{"key":"B10","unstructured":"Towards a Field for Collaborative Education Research: Developing a Framework for the Complexity of Necessary Learning2023"},{"key":"B11","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1002\/ajcp.12447","article-title":"A participatory, mixed methods approach to define and measure partnership synergy in long-standing equity-focused CBPR partnerships","volume":"66","author":"Coombe","year":"2020","journal-title":"Am. J. Commun. Psychol."},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/2332858419858635","article-title":"Negotiating trust, power, and culture in a research-practice partnership","volume":"5","author":"Denner","year":"2019","journal-title":"AERA Open"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3102\/01623737026001001","article-title":"Are we asking the right questions? Using cognitive interviews to improve surveys in education research","volume":"26","author":"Desimone","year":"2004","journal-title":"Educ. Eval. Pol. Anal"},{"key":"B14","unstructured":"DiamondJ.\n          Racial Equity and Research Practice Partnership 2.0: A Critical Reflection.2021"},{"key":"B15","author":"Farrell","year":"2021","journal-title":"Research-practice Partnerships Today: The State of the Field"},{"key":"B16","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1177\/109821400402500105","article-title":"Using collaboration theory to evaluate strategic alliances","volume":"25","author":"Gadja","year":"2004","journal-title":"Am. J. Eval"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1937","DOI":"10.1002\/jcop.22239","article-title":"Content validation of a quantitative stakeholder engagement measure","volume":"47","author":"Goodman","year":"2019","journal-title":"J. Commun. Psychol"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.17605\/OSF.IO\/YVGCN","author":"Henrick","year":"2023","journal-title":"Indicators of Research-Practice Partnership Health and Effectiveness: Updating the Five Dimensions Framework."},{"key":"B19","volume-title":"Assessing Research-Practice Partnerships: Five Dimensions of Effectiveness.","author":"Henrick","year":"2017"},{"key":"B20","unstructured":"HouseE. R.\n          The Logic of Evaluative Argument1977"},{"key":"B21","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1111\/emip.12393","article-title":"When should i use a measure to support instructional improvement at scale? The importance of considering both intended and actual use in validity arguments","volume":"40","author":"Ing","year":"2021","journal-title":"Educ. Measur"},{"key":"B22","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1146\/annurev.publhealth.19.1.173","article-title":"Review of community-based research: assessing partnership approaches to improve public health","volume":"19","author":"Israel","year":"1998","journal-title":"Ann. Rev. Publ. Health"},{"key":"B23","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1177\/1757975911412402","article-title":"Partnership tools for health promotion: are they worth the effort?","volume":"18","author":"Joss","year":"2011","journal-title":"Glob. Health Promot"},{"key":"B24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-3-319-17727-4_122-1","article-title":"\u201cDesign-centric research-practice partnerships: three key lenses for building productive bridges between theory and practice,\u201d","volume-title":"Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy","author":"Kali","year":"2018"},{"key":"B25","first-page":"131","article-title":"\u201cContent-related validity evidence in test development,\u201d","volume-title":"Handbook of Test Development","author":"Kane","year":"2006"},{"key":"B26","doi-asserted-by":"publisher","first-page":"769","DOI":"10.2753\/PMR1530-9576350410","article-title":"Measuring the performance of public-private partnerships: a systematic method for distinguishing outputs from outcomes","volume":"35","author":"Koontz","year":"2012","journal-title":"Publ. Perform. Manag. Rev"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1558689816633309","DOI":"10.1177\/1558689816633309","article-title":"Development of a mixed methods investigation of process and outcomes of community-based participatory research","volume":"12","author":"Lucero","year":"2016","journal-title":"J. Mixed Methods Res."},{"key":"B28","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1111\/1468-0009.12458","article-title":"Measuring community-engaged research contexts, processes, and outcomes: a mapping review","volume":"98","author":"Luger","year":"2020","journal-title":"Milbank Quart"},{"key":"B29","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1177\/0193841X8500900504","article-title":"Stakeholder-based evaluation and value judgments","volume":"9","author":"Mark","year":"1985","journal-title":"Eval. Rev"},{"key":"B30","doi-asserted-by":"publisher","first-page":"S151","DOI":"10.1016\/j.amepre.2008.05.020","article-title":"Measuring collaboration and transdisciplinary integration in team science","volume":"35","author":"Masse","year":"2008","journal-title":"Am. J. Prev. Med."},{"key":"B31","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1007\/s11423-016-9453-6","article-title":"Measuring experiences of interest-related pursuits in connected learning","volume":"16","author":"Maul","year":"2016","journal-title":"Educ. Technol. Res. Dev"},{"key":"B32","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1145\/3477607","article-title":"Exploring research practice partnerships for use in K-12 computer science education","volume":"12","author":"McGill","year":"2021","journal-title":"ACM Inroads"},{"key":"B33","doi-asserted-by":"publisher","first-page":"13","DOI":"10.2307\/1176219","article-title":"The interplay of evidence and consequences in the validation of performance assessments","volume":"23","author":"Messick","year":"1994","journal-title":"Educ. Research"},{"key":"B34","volume-title":"Qualitative Data Analysis: An Expanded Sourcebook","author":"Miles","year":"1994"},{"key":"B35","unstructured":"\u201cBayes nets in educational assessment: where do the numbers come from?\u201d437446\n            MislevyR.\n            AlmondR.\n            YanD.\n            SteinbergL. S.\n          Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence1999"},{"key":"B36","doi-asserted-by":"crossref","DOI":"10.1002\/j.2333-8504.2003.tb01908.x","volume-title":"A Brief Introduction to Evidence-Centered Design.","author":"Mislevy","year":"2003"},{"key":"B37","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-52246-3","volume-title":"Assessing Model-Based Reasoning using Evidence-Centered Design.","author":"Mislevy","year":"2017"},{"key":"B38","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1111\/j.1745-3992.2006.00075.x","article-title":"Implications of evidence-centered design for educational testing","volume":"25","author":"Mislevy","year":"2006","journal-title":"Educ. Measur"},{"key":"B39","unstructured":"RPP Effectiveness and Health Toolkit Website2024"},{"key":"B40","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1007\/s10488-013-0528-y","article-title":"Purposeful sampling for qualitative data collection and analysis in mixed method implementation research","volume":"42","author":"Palinkas","year":"2015","journal-title":"Admin. Pol. Mental Health"},{"key":"B41","doi-asserted-by":"publisher","first-page":"1090198120921188","DOI":"10.1177\/1090198120921188","article-title":"Engage for equity: development of community-based participatory research tools","volume":"47","author":"Parker","year":"2020","journal-title":"Health Educ. Behav."},{"key":"B42","doi-asserted-by":"publisher","first-page":"800","DOI":"10.1080\/13511610.2023.2191827","article-title":"A relations-based evaluation framework to support collaborative research and innovation (R&I) networks","volume":"36","author":"Pisacane","year":"2023","journal-title":"Innov. Eur. J. Soc. Sci. Res"},{"key":"B43","doi-asserted-by":"publisher","first-page":"2042682","DOI":"10.1080\/10627197.2022.2042682","article-title":"Disrupting white supremacy in assessment: toward a justice-oriented, antiracist validity framework","volume":"2022","author":"Randall","year":"2022","journal-title":"Educ. Assess"},{"key":"B44","unstructured":"SchneiderM.\n          Developing an Evidence Base for Researcher-Practitioner Partnerships.2018"},{"key":"B45","unstructured":"ScholzC.\n            LaTurnerJ.\n            BarkowskiE.\n          Tool for Assessing the Health of Research-Practice Partnerships2021"},{"key":"B46","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/S0149-7189(03)00029-6","article-title":"Instrument for evaluating dimensions of group dynamics within community-based participatory research partnerships","volume":"26","author":"Schulz","year":"2003","journal-title":"Eval. Progr. Plan"},{"key":"B47","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1111\/j.1745-3992.1997.tb00585.x","article-title":"The centrality of test use and consequences for test validity","volume":"16","author":"Shepard","year":"1997","journal-title":"Educ. Measur"},{"key":"B48","doi-asserted-by":"publisher","first-page":"56","DOI":"10.3389\/fpubh.2016.00056","article-title":"Community case study article type: criteria for submission and peer review","volume":"4","author":"Smith","year":"2016","journal-title":"Front. Publ. Health"},{"key":"B50","first-page":"423","article-title":"\u201cMeasurement for improvement,\u201d","volume-title":"The Foundational Handbook on Improvement Research in Education","author":"Takahashi","year":"2022"},{"key":"B51","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1080\/1743727X.2022.2097218","article-title":"Toward a critical race RPP: how race, power and positionality inform research practice partnerships","volume":"45","author":"Tanksley","year":"2022","journal-title":"Int. J. Res. Method Educ"},{"key":"B52","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1093\/jopart\/mum036","article-title":"Conceptualizing and measuring collaboration","volume":"19","author":"Thomson","year":"2009","journal-title":"J. Publ. Admin. Res. Theor"},{"key":"B53","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1017\/cts.2019.402","article-title":"Measuring quality and outcomes of research collaborations: an integrative review","volume":"3","author":"Tigges","year":"2019","journal-title":"J. Clin. Transl. Sci"},{"key":"B54","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10551-015-2756-4","article-title":"Enhancing the impact of cross-sector partnerships: four impact loops for channeling partnership studies","volume":"135","author":"Van Tulder","year":"2016","journal-title":"J. Bus. Ethics"},{"key":"B55","doi-asserted-by":"publisher","first-page":"8959048221130353","DOI":"10.1177\/08959048221130353","article-title":"Research-practice partnerships in pursuit of racial justice in schools: navigating a hostile sociopolitical climate","volume":"37","author":"Villavicencio","year":"2022","journal-title":"Educ. Pol."},{"key":"B56","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1177\/1090198119897075","article-title":"Engage for equity: a long-term study of community-based participatory research and community-engaged research practices and outcomes","volume":"47","author":"Wallerstein","year":"2020","journal-title":"Health Educ. Behav"},{"key":"B57","doi-asserted-by":"publisher","first-page":"101013","DOI":"10.1016\/j.stueduc.2021.101013","article-title":"Research-practice partnerships 'on the ground:' exploring partnership work in urban schools under pressure to improve","volume":"70","author":"Weddle","year":"2021","journal-title":"Stud. Educ. Eval."},{"key":"B58","volume-title":"Research-Practice Partnership Effectiveness: Which Strategies Matter for Impact?","author":"Wentworth","year":"2018"}],"container-title":["Frontiers in Research Metrics and Analytics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2024.1210547\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T07:04:56Z","timestamp":1723446296000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2024.1210547\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,12]]},"references-count":57,"alternative-id":["10.3389\/frma.2024.1210547"],"URL":"https:\/\/doi.org\/10.3389\/frma.2024.1210547","relation":{},"ISSN":["2504-0537"],"issn-type":[{"value":"2504-0537","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,12]]},"article-number":"1210547"}}