Myths About International Test Score Comparisons

A Forum — June 2, 1995

Test score comparisons have continually been used within education policy to help determine the quality of education provided by schools, school districts, states, and even entire nations.  Test score comparisons shape legislation and affect national education standards, education finance, and management structures. Further, test scores are often blamed for the perceived failures of the nation's economy.  Dr. Iris C. Rotberg, Program Director for Research, Evaluation and Dissemination at the National Science Foundation, argued that this use of test score comparisons is "highly misleading as an indicator of the quality of education" provided to the nation's children.

Rotberg contended that the propagation of test score comparisons is due to "a series of myths" surrounding the importance of standardized testing.  In addition to outlining what she sees as the five most prevalent of these myths, Rotberg addressed the appropriate and inappropriate uses of testing, continually highlighting the importance of viewing test scores not as a direct measure of quality and teacher accountability but only as a guide for determining the ability of individual students.

The first "myth" Rotberg discussed is that test score comparisons between nations, states, and schools provide valid measures of quality of education.  This myth fails to recognize that there are substantive differences in education providers across regions.  It also overlooks the varying populations of students who take standardized tests across these regions, poverty rates, and the impact of different curricula.  For example, the United States has the highest school attendance rate in the world.  As a result, a greater proportion of the population takes standardized tests, invariably lowering aggregate test scores.  Unlike the United States, many other nations place less value on the education of disadvantaged children, and may experience inflated test scores since only their "best" students are taking standardized tests.

Studies have shown that states with the lowest proportion of students taking the SAT typically have the highest SAT scores: such states' test scores come from a smaller number of more select students, showing that schools can raise scores simply by excluding students.  As a result, Rotberg said, test scores cannot be considered a viable measure of the quality of education schools can provide.

Curriculum also has a clear impact on test scores.  Countries, states, and districts that emphasize a particular subject, such as math, will likely produce students who score higher in that subject than students from areas that promote a more diverse curriculum.  These higher test scores do not necessarily signify a higher quality of education, but only a difference between the type of education these jurisdictions choose to provide.

Rotberg allowed that test scores can be used to compare student achievement levels.  While not a valid measure for comparing the ability of schools to provide a quality education, test scores can help compare the ability and achievement of individual students.

The next "myth" Rotberg identified was that test scores show that the quality of America's schools has declined and thus made us less economically competitive as a nation.  Americans, Rotberg argued, are very nostalgic, and tend to think that schools in America's past were better and more rigorous than today's.  This, however, ignores the dramatic strides made in American education during the 20th century.  In 1940, only 38 percent of Americans 25- to 29-years-old possessed a high school diploma.  By 1993, that figure had climbed to 87 percent.  Over the same time period, the percentage of people receiving a bachelor's degree increased from six to 24 percent.  In addition, test scores today are at least as good, if not better than, scores in the 1960s and 1970s, and verbal test scores have gone up for all minority groups during this time.

Next, Rotberg addressed a third myth: "We can fix our schools by administering more tests."  But new tests are most likely to be mandated for low-income children -- those with the lowest levels of achievement.  Requiring them to take new tests will not lead to programs and policies that foster student development or create better students, according to Rotberg.  Instead, they will be trained only in how to take these tests, focusing on rote learning in an attempt to improve test scores, and neglecting the areas where they need more help -- problem-solving, communication, and self-esteem.

Rotberg then addressed a related myth: Problems with current standardized testing programs can be solved with more innovative tests, like portfolios and performance assessments.  This, according to Rotberg, still ignores that fact that test score comparisons cannot be used to determine the quality of education a school, district, state, or nation provides. The more innovative tests are harder to evaluate than multiple choice tests and have no constant measure across scores, thus further threatening the validity of test score comparisons.

Others, however, promote the use of such tests for the instructional benefits they produce.  They argue that to prepare students for tests that measure problem-solving and writing ability, teachers must develop new teaching methods. Rotberg, however, argued that encouraging additional problem solving and writing techniques in the classroom while discouraging a new wave of tests would better educate students but avoid the tremendous costs and time involved in grading and scoring tests that have minimal relevance to educational quality.

Finally, Rotberg attacked the myth that "we can compensate for poor performance by increasing testing requirements."  This myth implies that financial investment in education is irrelevant, and that requiring students to take more tests will inspire them to work harder and improve their academic performance.  Rotberg argued that this is a fallacy. She emphasized that improved performance is a function of smaller student/teacher ratios, equity in education funding and providing children with clean and safe environments within which to learn.  Increasing testing requirements "encourages quick-fixes and ignores real problems.  We need to do real work."

Discussion Period

The first point of discussion was the lack of a "high-stakes" test for students in the United States.  While tests such as the SAT are important factors in college acceptance, nations such as England and France employ tests that youth must pass if they wish to attend any college at all.  For students in the United States who perform poorly, college is often still an option.  This, according to Rotberg, is a function of America's wealth and its emphasis on diversity, and must be considered when discussing the use of test score comparisons.

The discussion also focused on how test score comparisons are attractive in our political system's "sound-bite" atmosphere, offering policy makers a quick, easy, and effective way to get a point across.  Rotberg responded that if used to compare achievement levels between individual students, test scores can effectively determine areas for improvement or additional support.  Using test scores as indicators of quality of education, however, can condemn policies to failure, since inaccurate measurements will yield unstable policies.

The forum concluded with the question of how policy makers can determine educational quality without using test score comparisons.  Rotberg, as well as other forum participants, argued that the issues surrounding quality of education tell us much more about a school's quality than do test scores.  Quality education comes from caring parents and communities that are devoted to working with and caring for children.  Policy makers and the public already recognize this, and do not require tests to determine whether these traits are present in particular schools.  For education policies to be efficient and effective, policy makers must do what they already know must be done -- promote the development of quality schools and nurturing environments -- and not attempt to hide behind the myths of test score comparisons.

American Youth Policy Forum   1836 Jefferson Place NW,  Washington DC 20036
Telephone 202-775-9731      |      Fax 202-775-9733      |     Email aypf@aypf.org