This is the fifth post in our eight-part blog series on using research evidence under the Every Student Succeeds Act (ESSA). Check back every week for the latest installment. Also, be sure to sign up for our March 7 webinar, ESSA and Research Evidence: Opportunities and Challenges for States.
Ruth Neild
“Invest in what works, and stop doing what doesn’t!” This, in a nutshell, is the call to base policy and funding decisions on scientific research rather than anecdote, belief, or supposition. This commitment also underlies the evidence requirements in the Every Student Succeeds Act (ESSA), which outlines a hierarchy of causal evidence and, in some titles, requires use of “activities, strategies, and interventions” backed by more rigorous evidence.
But what does this mean in practice? To comply with ESSA, should schools plan a shopping trip for a new textbook series if their current materials do not have evidence of better outcomes compared to an alternative? Should they contract with a provider for a new program or two? Or could they use the resources they already have in a new way, such as by focusing on classroom instructional strategies supported by rigorous evidence?
Clearly, as states begin to develop their plans for implementing ESSA, the “what” in “what works” needs some unpacking.
The “what” in education is more than programs
Some people imagine the “what” in “what works” as standalone programs or program models intended to accomplish specific things. For example, they might think of programs with prescribed materials, training, and/or procedures – a curriculum, or a professional development program, or a whole-school improvement model. But in education, the choices that schools and districts make encompass not only programs and models but also products, policies, and even instructional strategies for teaching very specific content.
To illustrate: pause for a moment, and think of something that rigorous research has identified as producing positive student outcomes in education. Now, consider: did you think of a program – that is, a set of components that are intended to be implemented together, that may carry a brand name, and that may require copyrighted materials or technical assistance that need to be purchased? Or did you think of a model – that is, a general set of strategies for addressing a challenge in education, such as career academies? How about a policy – for example, offering additional compensation to encourage effective teachers to transfer to low-performing schools? Or perhaps what came to mind is a practice, such as using solved problems in algebra instruction or providing social skills training to improve behavior among preschoolers. Maybe you thought of a product, such as software intended to teach a particular concept. Or perhaps you thought of some combination of the above.
If this feels like a great blooming, buzzing confusion, take heart! There are some sensible steps to take to sort through the evidence. And there are tools, like the What Works ClearinghouseTM (WWC), that can make the journey easier for you. Below are four tips to help you get started.
Tips for assessing the evidence on programs, products, policies, and practices
If you’re considering a branded intervention, such as a curriculum, program, or product, start by looking for evidence on that particular intervention. Most of the interventions summarized in the Find What Works feature of the WWC are branded. Find What Works provides assessments from the WWC’s systematic reviews, which involve a comprehensive search for all of the literature on an intervention and a summary of evidence from effectiveness studies that meet WWC standards.
If there is no evidence on a particular branded intervention, seek evidence on the general approach. The branded intervention may be part of a general class about which there is evidence. For example, while there may not be evidence about a specific career academy program, the program may have the same key elements as other career academy models that have been found to have positive effects. A helpful evidence review by RAND makes this crosswalk for leadership programs by highlighting key elements of effective programs. The RAND report also notes a clarification by the U.S. Department of Education that the “label or brand attached to a program or intervention…is less important than the activities, strategies, and practices that constitute that program or intervention.”
You don’t necessarily have to buy stuff to meet ESSA requirements! Consider classroom practices supported by rigorous evidence. Find What Works is not the only place on the WWC website to learn about effective interventions. The most popular WWC products are practice guides, which are developed by panels of researchers and education practitioners and use a rigorous rating system to characterize the research evidence for each recommendation. Implementing the practice guide recommendations does not depend on having access to particular branded products. Many of the more than 20 WWC practice guides focus on instructional challenges such as improving student writing at the elementary or secondary level, teaching beginning reading, or providing algebra instruction.
To support educator implementation of the recommendations, the Regional Educational Laboratories have begun to develop additional materials for school-based professional learning communities (PLCs) based on the WWC practice guides. The two currently available PLC guides – on teaching academic content and literacy to English learners and beginning reading – include handouts, activities, videos, and notes for the facilitator. These materials, like the WWC practice guides and Find What Works, are all free of charge.
To plan for the future, assess current and potential instructional materials against strategies recommended in the WWC’s practice guides. It’s ideal to have rigorous evidence on whether a particular set of instructional materials improves student outcomes more than another reasonable alternative. But, lacking this evidence, educators can use the recommendations in the WWC practice guides to assess materials they are using or considering. For example, Regional Educational Laboratory Southeast has produced a free, downloadable rubric for assessing K-5 materials in reading/language arts, with each criterion tied to a recommendation in a WWC practice guide.
A final word: Making the best bets
Our understanding of effective (and ineffective) interventions has advanced rapidly in recent years. Still, rigorous evidence in education is best thought of as a guide to making the best bets with available resources. “What worked” in the past makes no ironclad promises for what will work in a new place, group, or time. Evidence, then, is a guide to help you — the education decision maker — narrow the field of choices, cut through the clamor of the market pitching new products and programs, separate fact from fad and whimsy, and zero in on the “activities, strategies, and interventions” that have some track record of improving important education outcomes. The key is the thoughtful educator or policymaker interpreting the evidence in a sensible way to learn what it might have to say to their own context.
Ruth Neild is Director, Philadelphia Education Research Consortium
