Overview
Increasingly, policymakers and practitioners are interested in ELOs for various reasons. This forum, the second in a four-part series on Extended Learning Opportunities (ELOs) for Older Youth sponsored by the W.T. Grant Foundation, provided new research findings from the Policy Studies Associates (PSA) evaluation of the Citizen Schools (CS) program, an ELO program that prepares middle school students for success in high school and beyond.
Citizen Schools complements classroom learning by engaging students in experiential learning apprenticeship projects led by adult volunteers and supported by a staff of professional educators. Evidence from the evaluation demonstrates that the program is increasing academic engagement and achievement of students in Boston; specifically, former participants selected high-quality high schools at a higher rate than nonparticipants; former participants had higher school attendance rates than nonparticipants and were more likely to pass their English and math courses; former participants were more likely to pass the high stakes standardized English exam and to score proficient or advanced; and former participants were more likely than matched nonparticipants to be on-track to graduate from high school on time.
Citizen Schools is a growing national network of after-school education programs for students in 6th, 7th, and 8th grades that partners with middle schools to expand the learning day for low-income children. Citizen Schools programs complement classroom learning by engaging students in experiential learning projects led by adult citizen volunteers and supported by a staff of professional educators. Today, CS operates in middle schools in seven states, serving 4,500 students on 40 campuses and engaging 3,800 volunteers. There are plans to expand CS to 60 campuses by 2012 involving 6,700 students and 5,600 volunteers.
Evidence from PSA’s evaluation of CS in Boston demonstrates that the program is increasing academic engagement and achievement of students; specifically, former participants selected high-quality high schools at a higher rate than nonparticipants; former participants had higher school attendance rates than nonparticipants and were more likely to pass their English and math courses; former participants were more likely to pass a high stakes standardized English exam and to score proficient or advanced; and former participants were more likely than matched nonparticipants to be on-track to graduate from high school on time.
Eric Schwarz, Founder, President & C.E.O. of Citizen Schools, framed the forum by providing a sense of what CS is and how it evolved. In 1983, A Nation at Risk created a high level challenge to the country to improve public education for all students. Since then, a lot of attention has been placed on testing for achievement, reforming schools, and increasing the rigor of curriculum, however, even with increased attention directed towards reforming public schools, high school graduation rates have not improved and in some cases, have even gone down.. Nationally, high schools graduate 69-72% of youth and only 50% of low-income youth. Schwarz said the picture is bleak. The National Assessment of Educational Progress [NAEP] shows incremental success has occurred with younger children, but it shows flat results on academic measures for older students. There are, of course, islands of excellence, but overall, the results are not good. Part of the problem, he continued, is that we still operate on the old paradigm of school occurring six hours per day, 180 days per year on an agrarian calendar, with 25-30 students sitting in a classroom with one teacher. Schwarz explained that our country is also losing ground with regard to helping students develop 21st Century skills needed by the business community, including higher order thinking, team work, and creativity, which are not usually taught in school. Schwarz said that CS re-imagined what the school day could look like in order to help students develop some of these skills and asked: What if students could work in teams with other students and participate in internships and externships? CS decided to take on this design challenge in 1995.
Within a year, the basic elements of CS were in place: after-school programs operating out of Boston Public School buildings across the city, apprenticeships taught by volunteer Citizen Teachers, community explorations, field trips, team-building activities, and homework and school skills. CS expands learning time by 30-40% and adds 400 more hours of structured learning time per year in school and out of school. CS takes place almost every week of the school year and operates four-five days per week, adding nearly three hours per day. CS helps parents and students navigate through school and handles immediate needs, such as homework, and future needs, such as navigating the transition to high school and college.
Schwarz explained that there are limits to what even the best teaching force can do and promoted the concept of having access to a second shift of teachers, provided by AmeriCorps members and other additional caring adults in students’ lives. CS employs full- and part-time paid staff, many of whom make a two-year commitment, and some teachers stay and progress to lead teachers.
Citizen Schools provides two courses that run for 11 weeks each semester. On Tuesdays and Thursdays, CS students participate in “apprenticeships” that are taught by Citizen Volunteers. Citizen volunteers range from Google employees to Wall Street attorneys. . It is in these apprenticeships that students work on real-life projects such as developing a power point presentation on college costs, making furniture, or creating a video game or an architectural design. Students are coached in each project by an expert in that field. Apprenticeships are followed by a “WOW” event where students present their work to adults and staff. One benefit of CS is that students are exposed to more caring adults through the apprenticeship program.
During the weekly ELO program, students have a daily transition and snack period of 15 minutes followed by math leagues, homework, and/ or organization and study skills sessions for 60 minutes, followed by team time, apprenticeship and/or experiential explorations for 90 minutes.
At several CS sites, the last period of the school day, often math class, is an overlap between the regular day and after-school program. Sometimes after-school teachers sit in on the last period class to reinforce teachers in the classroom, provide extra help, help coordinate homework, and better align their afternoon curriculum.
Elizabeth R. Reisner, Principal, Policy Studies Associates (PSA) provided the background and context of the findings of the PSA evaluation of CS. The study was designed to determine core elements of CS, and how participants in the CS program performed on indicators that measured academic skills and high school and college success. Specifically, the evaluation asked the question: Do enriched learning experiences build skills for long-term success?
For five years, PSA has tracked the progress of CS participants in Boston, using a matched comparison group. The evaluation covered 6th, 7th and 8th grade students. The study was quasi-experimental in design and for 8th graders specifically, a matched comparison group was used. The matched comparison was achieved with close cooperation with the the Boston Public Schools. PSA has found that CS participants selected high-quality high schools more often than non-participants. CS participants also had higher levels of academic achievement and were more likely to be on track for high school graduation. PSA is currently analyzing early data to determine how many graduate from high school. Early indications are that former CS participants are graduating at higher rates than are the matched comparison students. Former CS participants are not part of an organized CS program in high school, although they may check in now and then with the middle school program.
Juliet Diehl Vile, Research Associate, Policy Studies Associates, provided additional information on the Boston study. Juliet explained that the high school findings were divided into three main categories: transition to high school, engagement and academic achievement in high school, and progress towards high school graduation. To measure transition to high school, PSA looked at the percentage of participants and matched nonparticipants enrolling in high quality high schools. Students in Boston have a choice of over 30 high schools, and one of the goals of CS is to help participants choose higher-performing and more rigorous high schools. High schools are categorized into three categories, based on qualitative and quantitative data. The study found that CS participants enrolled in high quality high schools at two times the rate of nonparticipants.
The PSA findings on engagement and academic achievement of former CS participants in high school showed positive outcomes, many of which were statistically significant. The percent of participants passing their 9th grade English class was 84% versus only 71% of matched nonparticipants (statistically significant). The percent of participants passing their 9th grade math course was 72% versus only 68% of matched nonparticipants (not statistically significant). The percent of participants passing the English Language Arts Massachusetts Comprehensive Assessment System (MCAS) was 94% versus 86% of matched nonparticipants (statistically significant). The percent of participants passing the math MCAS was 86% versus 81% of matched nonparticipants (not statistically significant).
To measure former CS participants’ progress towards high school graduation, PSA measured whether students were on track to graduate and whether or not they had failed any core subjects. Findings were very positive and were even greater for the high exposure group. Sixty-two percent (62%) of former CS participants were on track to graduation and had not failed any core course versus 52% of matched nonparticipants. For former CS participants that attended CS for at least two years and came 60% of the time or more, 71% met the same measurement versus 52% of matched nonparticipants. For lower exposure former CS participants, 55% met the same measurement versus 52% of matched nonparticipants.
Elena Kennedy, Second Year Citizen Schools Teaching Fellow and Coordinator for the Citizen Schools 8th Grade Academy program in Houston, Texas, provided a detailed description of what participation in Citizen Schools looks like. Kennedy supervises the curriculum and instruction in the 8th Grade Academy (8GA) College Awareness and Access Program and coordinates high school choice activities for families and students across three Houston Middle Schools. Kennedy provided an example of a typical participant, Kelly, whose parents are from Bolivia. Her father works two jobs as a licensed electrician and a busboy. Her parents’ dream is for her to finish college. Both her older brothers work at Starbucks so that she can go to college. Although Kelly appeared not to be that interested in school, she attended CS every day and went on every field trip. She chose a good high school — Challenge Early College High School – which cooperates with the local community college to provide students with an Associate’s degree in four years. She also participates in a tennis club and has earned A’s in her first semester of high school. Kennedy said that the students named their CS program “The Future,” because they see how it helps them prepare for college. The 8GA has grown from 13 students to over 70 students.
Three primary areas in which students in 8GA receive instruction and show positive outcomes are high school choice and the transition from middle school to high school, understanding college culture (in addition to following a college prep curriculum during the school day) and career exposure.
To prepare for this important choice of high schools, 8GA students visit and research high-quality high schools. In partnership with Univision Radio, students wrote and produced PSA’s about the importance of choosing a strong high school that were played on the radio and in the middle school, and students also created power point presentations about Houston high schools for other students
In 2007-2008, CS data showed that although only 9% of Houston 8GA students lived in school zones with top-tier high schools and 70% lived in zones with bottom-tier high schools, many chose top-tier schools instead (56%) with only 21% going to the bottom-tier schools. In fact, acceptance at a top-tier high school was 75% for participants versus 47% for nonparticipants. Students who reported being unsure of their high school choice decreased from 21% to 9%. Initially, no students thought they would go to private school and 14% ended up in private school.
Participation in high school transition activities is high for CS participants. For example, 100% of CS participants attended at least one high school open house versus only 31% of nonparticipants. Seventy percent (70%) of participants attended more than one high school event versus 11% of nonparticipants.
In terms of helping participants understand college culture, CS and middle school staff work together to foster a college bound culture on campus. 8GA students spend two hours per week on college awareness and culture, they practice for the SAT, they learn how to navigate high school courses including a familiarity with the Texas College Prep Curriculum, they meet with a ninth grade counselor to discuss their plans for high school and beyond, and they take between eight and ten college trips per year. Program evaluations show that 8GA students are more knowledgeable about the SAT and courses needed for college than their peers. Seventy percent (70%) of 8GA participants were knowledgeable about the SAT versus 30% of non 8GA participants. Eighty-eight percent (88%) of 8GA participants were knowledgeable about the Texas College Prep Curriculum versus 48% of non 8GA participants. CS also holds college awareness events. CS provides benefits for all students and parents in a school, not just those in the program. For example, as part of building a college culture, Grady Middle School held an open house that was attended by 200 families, which was the largest gathering of parents ever held at the school, the previous record being 30 parents.
A career day was also sponsored by 8GA,where CS participants learned that many careers require at least a Bachelor’s degree. The 8GA Career Fair engaged over 40 professionals in knowledge sharing and mentoring students. As part of the 8GA Careers Unit, students learn how to write resumes, dress professionally, and prepare for job interviews.
The CS program evaluations by staff also show improved school engagement for 8GA students. Pre- and post-tests were used to determine the level of student engagement. Findings indicated that CS students were more engaged in school over time than their peers.
Schwarz wrapped up the discussion by emphasizing that eighth grade should be seen as a bridge to college. He indicated that some may feel this is too early, but that middle school is a good time to help youth see high school as relevant and to make college visits. Many eighth grade students make important choices about which high school to attend, which classes to take in high school and they also begin to prepare for the PSAT and SAT. CS helps demystify the college pathway. Eighth grade is also an important time to help young people develop career goals.
Schwarz also said that CS supports the Generations Invigorating Volunteerism and Education (GIVE) Act, which amends the National and Community Service Act of 1990 (NCSA) and the Domestic Volunteer Service Act of 1973 (DVSA) as programs that support their volunteers. CS also supports the Serve America Act, which would provide grants to school districts to support tutors, mentors, and citizen teachers. CS is also interested in policy that tries to extend the school day, such as a plan in Boston to extend the school day by 30% for all students in hopes of cutting the math achievement gap in half.
Finally, Schwarz stated that the federal government should think like the best philanthropists and “grow what works.” Incremental government money should be added to the most effective programs, multiplying and deepening their effect, he said. Government can also work with philanthropy to help support the best use of the stimulus package.
Highlights from the Question and Answer Session
The first question related to how after-school programs can be designed to overlap with the school schedule. The answer was that this is being piloted on a few campuses. It is very helpful for after-school staff to sit in on classes during the day and sometimes teamteach the last class of the day. After-school staff will then be aware of the homework assignments and school events, they can improve their outreach to parents, better align lesson plans and be seen as educational partners. This also makes the division of the day between regular school and after-school almost invisible to the students.
The second question asked about how citizen volunteers are trained, particularly about how they learn to understand young people and respect the fact that young people have a lot to offer. The reply was that citizen volunteers receive 4-5 hours of formal training, a 50-80 page manual regarding CS’s approach to experiential learning, formal interactions with other staff, and observations by other staff. Schwarz added that, due in part to their relationship with AmeriCorps, most or all staff members are college graduates with a few years of experience already. Many of them make a two year commitment to CS. Staff members can gain a lot of experience and move into their first supervisory jobs in roles such as Campus Director which is like an Assistant Principal. Some jobs pay in the $40-50,000 range and may entail leading 8 staff people and 100 plus children. CS takes staffing very seriously. While these staff salaries make programs expensive, they are less expensive at scale and it is important to invest in quality to get results.
The third question asked if CS plans to expand to high schools. Schwarz replied that he felt the concept could work in high school, but that CS will not go in that direction. He feels that middle schools need lots of attention and they will continue to refine and expand the concept at the middle school level.
The fourth question asked why evaluations often do not measure 21st Century skills if we all acknowledge these are so important. The reply was that research tends to look at things like math and English test scores and the evaluation used data already collected by Boston Public Schools. PSA is looking into measuring 21st Century skills such as using general problem solving items from the Program for International Student Assessment (PISA), creative logic solving problems, and other measures.
Another question regarded the importance of parent outreach, partnerships and supports, finding out about financial aid and working with Hispanic parents. Schwarz indicated that parents are contacted weekly or bi-weekly on a regular basis. Attempts are made to match staff members who speak languages in addition to English with parents who speak the same language. A question was asked about how CS is funded. Schwarz responded that CS receives both private and public funding, with more currently coming from private sources including many major corporations. Ideally, CS will become more publicly funded as it expands. At this time, 12-14% of all Boston 6th graders participate in CS, and the program is expanding in Houston, Redwood City, California, and elsewhere. When at scale, costs go down so that if there are 100-150 children on a campus, the cost is about $1,800-$2,200 per student per year. Schwarz said that by adding 10-15% more cost to the school day, students get 40-45% more time with strong educational results. The last question was how CS might compete or even conflict with other expanded learning opportunities such as Girls Inc. which has 300 affiliates. Schwarz acknowledged this possibility. He cautioned against expansions of programs that could simply lead to longer school days by paying existing teachers for more of their time. The added value of CS is the concept of first and second shift teachers, teaching academics differently while still aligning with the school day curriculum and using the school building.
Elena Kennedy, is a Second Year Teaching Fellow and coordinator for the Citizen Schools 8th Grade Academy program in Houston, Texas. In her role, Elena supervises the curriculum and instruction in the 8th Grade Academy College Awareness and Access Program, and coordinates High School choice activities for families and students across three Houston Middle Schools. Prior to joining Citizen Schools, Elena worked with Child Trends, a non-profit research center in Washington DC, which studies children at every stage in development. At Child Trends, Elena worked on projects studying parenting and family dynamics as well as projects regarding out-of-school time education and evaluation. Elena received her bachelor’s degree in Psychology from Middlebury College in 2006, and she will receive her master’s degree in Education from Lesley University in May of 2009.
Elizabeth R. Reisner, is one of two founders and managers of Policy Studies Associates, Inc., a Washington-based firm that conducts research and evaluation in education and youth development, specializing in the assessment of strategies to improve student learning in the elementary and secondary grades and to enhance the effectiveness of out-of-school time programs for children and youth. Ms. Reisner serves as principal investigator of several PSA evaluations of out-of-school time initiatives, including those sponsored by the New York City Department of Youth and Community Development, New Jersey After 3, Citizen Schools, and Save the Children /U.S. Programs. In addition, she serves as principal investigator of the evaluation of the middle-school initiative of the New York City Beacons. She recently served as co-principal investigator of a national research study that measured the effects of high-quality after-school programs on the cognitive and social development of disadvantaged children and youth.
Ms. Reisner served as principal investigator of the evaluation of the New Century High Schools in New York City. Ms. Reisner and PSA colleagues are currently involved in planning the evaluation of New York City’s investments in high school reform that have been supported by the Gates Foundation.
Ms. Reisner has worked with several Washington-area nonprofit organizations in their identification of program outcomes and the development of measurement systems to track progress toward the achievement of intended results. PSA is currently working with the New York City Department of Youth and Community Development to track youth outcomes among all youth served by the agency. She serves on the evaluation advisory committees of several private nonprofit youth-serving organizations and of the Edna McConnell Clark Foundation.
Eric Schwarz, is the Co-Founder, President & CEO of Citizen Schools, a leading education nonprofit that partners with middle schools to expand the learning day for low-income children. Citizen Schools has been awarded Fast Company’s Social Capitalist Award and the Skoll Foundation’s Award in Social Entrepreneurship. The organization serves 4,500 students and engages 3,800 volunteers across seven states.
Schwarz served on the Massachusetts Board of Elementary and Secondary Education’s Task Force on 21st Century Skills, the Center for American Progress working group on Expanded Learning Time, the transition team of Massachusetts Governor Deval Patrick, and New Profit, Inc.’s Social Entrepreneur Advisory Board. He is the author of “Realizing the American Dream: Historical Scorecard, Current Challenges, Future Opportunities”, and the co-editor of The Case for Twenty-First Century Learning.
Previously, Schwarz served as a Public Service Fellow at Harvard University and Vice President at City Year. Schwarz earned his B.A. at the University of Vermont and his Masters in Education at Harvard University.
Juliet Diehl Vile, is a Research Associate with Policy Studies Associates, Inc., a Washington-based firm that conducts research and evaluation in education and youth development. Ms. Vile is experienced in education evaluation and data analysis, including the analysis of local, state, and national education survey and assessment data. At PSA, Ms. Vile leads data analysis and manages evaluations in the areas of after-school programming, literacy, and higher education for disadvantaged youth and adults. She has conducted work for Citizen Schools, New York City Department of Youth and Community Development, Center for After-School Excellence, and Cornerstone National Literacy Initiative. Prior to joining PSA, Ms. Vile was a researcher with American Institutes for Research, Education Statistics Services Institute. Ms. Vile holds a B.S. in Industrial and Labor Relations from Cornell University and an Ed.M. in Education Policy and Management from the Harvard Graduate School of Education.
Photos Taken During the Forum
Eric Schwarz, is the Co-Founder, President & CEO of Citizen Schools
Elena Kennedy Juliet Diehl Vile Elizabeth R. Reisner
- AYPF ELO Executive Summary PDF Version
- AYPF ELO Matrix Pages 152-164
- Citizen Schools Fact Sheet
- Citizen School 2008 Evaluation Executive Summary
Presenters
Elena Kennedy
Citizen Schools
8th Grade Academy
2503 Robinhood St.
Houston, TX 77005
713.529.4006
Elizabeth R. Reisner
Principal
Policy Studies Associates
1718 Connecticut Ave., NW
Suite 400
Washington, DC 20009
202.939.5326
Eric Schwarz
Founder, President & C.E.O. of Citizen Schools
Citizen Schools
308 Congress St., 5th Fl
Boston, MA 02210
617.695.2300
Juliet Diehl Vile
Research Associate
Policy Studies Associates
1718 Connecticut Ave., NW
Suite 400
Washington, DC 20009
202.939.5326
