Loretta Goodwin, AYPF Deputy Director
Program officer: “Would you be willing to have your study tours evaluated?”
Me: “Sure! We love feedback and figuring out ways to improve!”
I know how tough it is to get third-party evaluations of work funded, so I was thrilled when the Hewlett Foundation recently funded AYPF to evaluate the 11 Deeper Learning study tours we have conducted between 2010 and 2017. (These tours are two- to three-day field trips for small teams of policy leaders and educators, combining presentations of material with site visits to schools and opportunities for networking.) Doing the evaluation fit with one of our core values focused on continuous education, believing as we do in providing ongoing and continuous high-quality, innovative professional development and learning opportunities for the policy community and for our own staff.
This year-long journey working with WestEd, which conducted the evaluation, proved highly instructive for our work. Their feedback informed not just our work related to providing policy leaders with information on schools engaged in deeper learning, but our events overall. It was great to learn our events are very well-received and have been instructive for policy leaders. But more useful were the recommendations for improvement. I’ll share some highlights here – free of charge – that might help you make your next professional learning experience a deeper learning one, whether it’s going along on a study tour, or organizing one yourself.
Before noting some of the evaluation’s specific topics and recommendations, it’s helpful to think about why you’d want to bring people (be they policymakers, business leaders or educators) along on a tour, and what you expect to happen following the tour. Or, if you’re going on a tour, think about your goals and what actions you are being motivated to take following your learning journey. WestEd evaluators provided a useful framework for us to think about tour goals, noting four stages for understanding behavior change: precontemplation, contemplation, preparation, and action. (This is the Transtheoretical Model of Change.) As we organize tours, our goal is that, having heard from policy leaders and visited schools, participants will move from contemplating change to actually taking action.
Based on examining feedback on our previous tours, accompanying us on two tours, interviewing past tour participants, and talking to AYPF staff, our WestEd evaluators provided the following feedback (highly condensed for blog purposes!). The pie charts show the percentage of participant beliefs, attitudes and comments that reflect positively on their experience for that specific topic.
1. Select and Prepare Participants Carefully
These tours work best when we bring together people with diverse backgrounds, experiences, and roles, a practice we’ll continue. Once selected, we provide participants with preparation materials, including draft agendas and information on the sites we’ll be visiting. In the future we’ll work on better connecting these preparatory materials to documents that will be used for reflection on participant learning during the tour. We’ll also examine ways to conduct pre-trip needs assessments, to help participants better focus their learning.
2. Provide High-quality Tours
Participants acknowledged that our tours were rich and rewarding experiences, with speakers setting the tone for discussion and helping to frame conversations. They appreciated the time for guided reflection and networking. Suggested improvements centered on providing more time during school visits for participants to interact with administrators, teachers and students. They also wanted to meet more students who’d graduated from high school, to hear more about how well their high school had prepared them for postsecondary options. Critically, participants wanted more help in teasing out valuable components from the school models presented, to help them incorporate some key practices into their own work. Unsurprisingly, they want to focus more on learning about what the policies and processes are at the teacher and administrator level that enable deeper learning.
3. Improve Follow-Up and Provide Additional Resources!
After the trip, we provide participants with resources, including panelist presentations and information on the school sites. They also liked receiving, a few weeks after the trip, the helpful reminder of a self-addressed postcard on which they’d shared the next action steps they committed to take, based on what they learned. Recommendations for improvement centered on providing participants with more opportunities to reflect on how the study tour would affect their work. We will also consider hosting a follow-up webinar to learn how participant work is being influenced by new knowledge from the trip. We’ve also been encouraged to follow up with state team members to see what actions they were taking and work with partner organizations to give participants resource tool kits. These kits would serve as a connection between the tour experience and common next steps or barriers.
4. Whatever Will Participants Now DO?
One thing is certain – following the tours, participants certainly discuss their experiences! They share with colleagues the research they’ve learned about and always chat about the students they have met. What we need to do more of is engage in purposeful, focused discussions on how policies can be enacted to support deeper learning, as our participants often struggle to connect what they see on the trips with policies they themselves could implement (or quit doing). We need to help them develop a plan of their next steps, and more immediately we have to help them develop the story they will tell about their learning. For future participants – you’re on notice – we’ll be asking at least one state team to provide an update a year after their study tour, letting us know what they’ve accomplished!
5. Keep Spreading Deeper Learning – and Adapting to Local Contexts!
Yes, folks leave our tours energized and inspired, and they do share school success stories and other publications related to deeper learning. They also appreciate that our tours are targeted at a specific topic, such as the intersection between deeper learning and career and technical education. Yet they clamor for ongoing support and guidance as they spread the concept of deeper learning. They long for exposure to ways they can be more effective with diverse student populations. They want to explore how deeper learning intersects with career pathway initiatives.
The vast majority of study tour participants had very positive feedback for us about the tours, buoying us as we get ready to conduct more of these great learning events. However, we know we can always continue to improve on these experiences, and that’s what we’re committed to doing, thanks to all the helpful recommendations from our colleagues at WestEd. Hopefully these recommendations fuel your own ongoing openness to feedback and desire for continuous improvement.
