Improving the Transition from Middle Grades to High Schools: The Role of Early Warning Indicators

Forum Brief - January 25, 2008

Low performing schools and low graduation rates continue to fuel nationwide high school reform efforts. Researchers have found that many young people drop out in the ninth grade, often for reasons associated with education factors such as failing grades and low test scores. While personal circumstances are a factor, the transition from eighth to ninth grade also presents adjustment challenges that can be harmful for students who are academically or socially unprepared. Students who drop out at this critical time follow identifiable patterns, and early warning indicators can signal problems if good data collection systems are in place.

This AYPF forum discusses early warning indicators, particularly the use of on-track indicators, that can identify potential high school dropouts as early as sixth grade. Educators can use on-track indicators to develop specific and intentional interventions to help these identified students. This forum is the first in a series examining various aspects of improving the transition from middle grades to high school and preparing students for success.

Robert Balfanz, Research Scientist, Center for Social Organization of Schools at Johns Hopkins University, discussed the major findings of his research in Boston and Philadelphia on keeping early secondary grade students on the graduation path. Results showed that students in high poverty school districts who successfully navigate 6th grade to 9th grade graduate from high school at a rate of 75% or higher.  Conversely, students from high poverty schools who become off-track between the middle grade to high school transition, graduate at a rate of 20% or lower.

Balfanz explained, “About 40% of eventual dropouts could be identified in the 6th grade and 75% by the 9th grade.” In his study, Balfanz followed three cohorts in large, urban, high poverty school districts from 6th grade through two years past expected graduation time. Balfanz stated, “Students get knocked off course in the early secondary grades by the ABC’s – Attendance, Behavior, and Course Failure. It makes sense. If you don’t come to school, you’re not learning. If you misbehave, you’re not learning. The result is course failure.” Balfanz found that approximately 80% of the Philadelphia 6th grade cohort that did not graduate from high school shared similar characteristics. The students that did not graduate attended school less than 80% of the school year, were suspended at some point, failed a math course, and/or failed an English course.

Balfanz further explained, “The critical threshold for attendance varies. Context matters. In Boston, students who attended less than 90% of the school year were in serious trouble, while in Philadelphia it was students who attended less than 80% of the time.  Being at the bottom part of attendance is the key thing. It’s where you fall in the distribution.” The variance in attendance between school districts suggests that the critical factor may not be total days missed, but being in the bottom of the attendance distribution.  Out of school suspensions and sustained mild misbehavior were also highly predictive for not graduating. “Having a grade for behavior told us something,” stated Balfanz.

Balfanz believes that course failure is a better predictor of graduation outcomes than test scores. “Students who fail mathematics, English, or any two courses in 6th grade are in trouble,” explained Balfanz.  Furthermore, when cross-referencing ABC data, 85% of 6th graders who failed English and 75% of those who failed math in Philadelphia also received a poor final behavior grade and/or attended school less than 80% of the time. 

We have underestimated the intensity and scale of the educational challenge in high poverty environments. It’s cognitively more challenging in middle grades. But in poor neighborhoods, there are other challenges that students face such as increased family responsibilities which tend to fall more on girls,” expressed Balfanz. He also believes that the onset of adolescence combined with concentrated, intergenerational poverty creates its own set of risk factors that are exacerbated by poor educational responses to these factors. Balfanz stated, “What do we do? Do we put good teachers there? No. We put our youngest teachers in these schools. We send in transient adults. We’re creating instability for our students when they already live with enough instability.” The result is a large population of early secondary grade students that begin to disengage from school and thus, attend school less, act out, and become disruptive in class.  Others just stop trying which also leads to course failure. The consequences of these actions result in low graduation rates, low achievement, increased likelihood of teenage pregnancy, and increased likelihood of involvement with the juvenile justice system. Balfanz noted, “We hope students will grow out of it, but in high poverty areas, they don’t grow out of it. As soon as kids are off-track, we need to aggressively approach these issues.”

Balfanz suggests that comprehensive, systematic, and sustained whole school reforms that specifically address attendance, behavior, and course performance are needed. He commented, “Limited reforms or partial implementation will lead to limited or partial success.” Using the results from his research, Balfanz described strategies that link early warning systems to tiered interventions:  

• Respond to the first signs that a student is falling off-track.

• Systematically apply school-wide preventative, targeted, and intensive interventions until the student is on-track.

• Use national service organizations (City Year, Americorps) to provide the person power to make available mentoring, tutoring, homework support, and to manage attendance and behavior programs at the needed scale for an affordable price.

• Use integrated student support providers (e.g. Communities in Schools) to bring in and monitor case-managed professional supports for the most needy students.

Furthermore, Balfanz suggests that, at the policy level, good legislation and proper funding is needed to address these issues to provide proper services for students, including a focus on middle school students. Currently, H.R. 3406, Success in the Middle Act of 2007, would provide grants to states to improve middle grade preparation for high school, if passed, and the Graduation Promise Act would enable needed reforms in the nation’s Dropout Factories.  Furthermore, Balfanz believes it would also be important to create an accurate measure of graduation rates to increase accountability. Balfanz concluded, “We know who’s going to drop out in the next five years with a high degree of certainty. We can identify with a high degree of certainty who they are. Let’s get to work.”

Elaine Allensworth, Co-Director for Statistical Analysis, Consortium on Chicago School Research at the University of Chicago, added to Balfanz’s assertions that graduation is a predictable event. Allensworth’s work relies on looking at students’ grades in the freshman year of high school. “We know graduation is related to many factors such as peers, family, and educational history. A lot of these issues are out of the control of the schools. However, [graduation] is a predictable event,” said Allensworth. In Chicago, Allensworth found that 81% of students that were on-track graduated while only 22% of students off-track after their freshman year graduated. Although students come in with varying achievement levels, 68% of students who tested in the bottom quartile graduated if they were on-track, while only 37% of students in the top quartile graduated if off-track.  Being on-track is a key indicator for graduation regardless of ability, race, or gender. The relationship between freshman year performance and graduation rates suggests that specific interventions can be implemented to improve graduation rates.

Looking at grade point averages (GPAs) after the freshman year, almost all students with less than a “D” average fail to graduate, while almost all students with a “B” average do graduate.  Allensworth stated, “Students with ‘D+/C-‘ GPAs could go either way. The students in the middle, these are the students that are seen as doing ok. We really need to start targeting these students.” Another indicator for risk of not graduating is the number of semester courses failed in the freshman year. As the number of semester courses failed increases in the freshman year, the percentage of students who graduate high school in four years decreases. Allensworth explained that graduation is determined by performance in high school courses and stated that other background factors can affect student performance, but background factors do not help predict graduation rates once we know students’ course grades.  Allensworth also believes that the focus on ACT preparation in Illinois detracts from valuable course instruction. “Schools are very focused on ACT test preparation because Illinois uses it as an accountability measure, but they are focusing on test strategies and breezing over information that will prepare students for deep problem-solving work, which is what they really need for college and for gains on the ACT,” stated Allensworth.

Allensworth reiterated that students’ behaviors and attendance are the key drivers of course failure and high grades, more so than academic ability as measured by test scores.  To improve grades, Allensworth believes we need to target students’ academic behaviors in their classes.  This will improve both high school graduation rates and college readiness levels. 

Craig Jerald, Independent Consultant, Break the Curve Consulting, discussed the importance of using data to develop strategic interventions to improve high school graduation rates. “Policymakers and educators often make decisions based on gut instinct, guesswork, or, even worse, myths and stereotypes,” stated Jerald. He believes that using data can help educators create more effective and efficient interventions to raise academic achievement and educational attainment.

Jerald is critical of past dropout prevention efforts and cites weak interventions as well as poor identification of at-risk students. Mathematica, Inc. developed a study that examined the risk factors used by federally-funded dropout prevention programs to identify students for interventions. The study found that dropout prevention programs “often serve more students who do not need dropout prevention services than students who do need them.” Jerald suggested that longitudinal cohort studies conducted by Balfanz and Allensworth are more valuable because they identify what warning signs to look for and when to look for them. Furthermore, Jerald believes that school systems could conduct their own “cohort analyses” to identify patterns in why their students drop out because warning signs can vary between different groups. This can help school systems target more effective and efficient interventions. Jerald explained, “The warning signs are not social, they are educational, so that means they are alterable.”

To improve interventions, Jerald discussed the need to reduce false-positives and false-negatives when identifying students for intervention. Reducing falsely identified students for intervention (those who would have graduated without intervention) would save money while reducing falsely unidentified students for intervention would help raise graduation rates. “There is no warning sign that is entirely predictive. There are false-positives and false-negatives,” stated Jerald.  Jerald supports the use of the on-track indicator because of its reliability and accuracy (approximately 85%) in identifying eventual dropouts. Using Chicago as a hypothetical, Jerald explained that even if the interventions using the on-track indicator turned out to be only 50% effective, the system would produce a 42% reduction of dropouts, increase the number of graduates by 48%, and increase the graduation rate by 22%. 

Jerald agreed with Balfanz and discussed the need for greater accountability of graduation rates. Jerald suggested that federal policy should provide support for states and districts to implement early warning data systems along with interventions. Jerald also believes the Success in the Middle Act could provide needed funds for early identification data systems.    

Highlights from the Question and Answer session

A question was asked about how conclusions can be drawn from correlation studies. Balfanz explained that their studies involved more complex data modeling than presented. “There is still an intermediate factor that educators can do something about. There are other factors that influence graduation rates but we can’t do anything about it,” stated Balfanz.

A question was asked about what helps high school students engage in their schoolwork.  Panelists agreed that students want to succeed and graduate and thus, schools need to create atmospheres of success. Furthermore, establishing the importance of attendance is key. Panelists suggested that interventions need to be manageable for adults and need to involve a combination of trained teachers and extra help. 

A question was raised about the amount of time that it takes a school to conduct their own data analysis. Panelists explained that time spent varies depending on the existing data system. Balfanz stated, “With the proper data system, it should take months, not years.” If a data system is not in place, funding and implementation of the data system must be taken into account.

 

Resources

Robert Balfanz: Keeping Early Secondary Grade Students on the Graduation Path Power Point Presentation

Elaine Allensworth: What Matters for Staying On-Track and Graduating Power Point Presentation

Craig D. Jerald: Stacking the Deck: Using Early Warning Data to Improve High School Outcomes Power Point Presentation

 

Presenters

Robert Balfanz is a Research Scientist at the Center for Social Organization of Schools, Johns Hopkins University. He is the Co-Director of the Talent Development Middle and High School Project, which is currently working with over 100 high poverty secondary schools to develop, implement and evaluate comprehensive whole school reforms. His work focuses on translating research findings into effective reforms for high poverty secondary schools. He has published widely on secondary school reform, high school dropouts, middle grade on-track indicators, and instructional interventions in high poverty middle and high schools. Recent work includes Locating the Dropout Crisis, with co-author Nettie Legters, in which they identify the number and location of high schools with high dropout rates, and What Your Community Can Do to End its Dropout Crisis. Dr. Balfanz is currently the lead investigator on a Keeping Middle Grade Students on the Graduation Track project in collaboration with the Philadelphia Education Fund. Dr. Balfanz is also the Co-Operator of the Baltimore Talent Development High School, a Baltimore City Public School System Innovation High School.  His recent work on middle and high school reform can be found at his website - www.gradgap.org

Elaine Allensworth is Co-Director for Statistical Analysis at the Consortium on Chicago School Research at the University of Chicago.  Her work focuses on the structural factors that affect high school students’ educational attainment, particularly the factors that affect graduation and dropout rates.  She was the lead author on a number of studies on graduation rates in the Chicago Public Schools, including The On-Track Indicator as a Predictor of High School Graduation (2005), What Matters for Staying On-Track and Graduating in Chicago Public High Schools (2007), and Ending Social Promotion: Dropout Rates in Chicago after Implementation of the Eighth-Grade Promotion Gate (2004).  She recently began a mixed-methods study of the transition to high school which will follow a cohort of students from eighth grade into their second year in high school.  This study looks at students’ perceptions of the challenges of high school, the school practices that can foster successful freshman-year performance, and the practices that can hinder students.  As a member of the CCSR postsecondary project, Elaine has been examining students’ transition out of high school, particularly the factors that affect performance on the ACT.  She is also leading several studies on the effects of high school curricula on students’ experiences in their classes and their academic outcomes. Elaine holds a Ph.D. in Sociology, and an M.A. in Sociology and Urban Studies from Michigan State University.  She was once a high school Spanish and science teacher.

Craig D. Jerald is an independent consultant specializing in education policy, communications, research, and practice.  His clients have included the Education Trust, Achieve Inc., the Center for American Progress, Education Sector, the Learning First Alliance, the National School Boards Association, the National Governors Association, and the Supreme Education Council of Qatar.

During 2007 Craig helped launch the Strong American Schools “ED in 08” campaign and served as its Director of Policy.  From July 2000 to July 2004, Craig was a Principal Partner at the Education Trust, where he worked extensively on issues related to teacher quality, accountability, federal education policy, and the practices of high-performing schools and districts.  As a Senior Editor at Education Week from 1996 to 2000, Craig founded and managed the organization’s research division and helped create and oversee Ed Week’s special annual reports series, Quality Counts and Technology Counts.  Craig also has worked at the U.S. Department of Education, and he began his career as a Teach for America recruit and middle school teacher in California’s Long Beach Unified School District.

 

American Youth Policy Forum   1836 Jefferson Place NW,  Washington DC 20036
Telephone 202-775-9731      |      Fax 202-775-9733      |     Email aypf@aypf.org