Research paper: Interpreting international test scores in light of social class differences

15 Jan

Moi wrote about international student rankings in Important Harvard report about U.S. student achievement ranking:

More and more, individuals with gravitas are opining about the American education system for reasons ranging from national security to economic competitiveness. In Condoleezza Rice and Joel Klein report about American Education, moi wrote:

The Council on Foreign Relations has issued the report, U.S. Education Reform and National Security. The chairs for the report are Joel I. Klein, News Corporation and Condoleezza Rice, Stanford University. Moi opined about the state of education in U.S. education failure: Running out of excuses Education tends to be populated by idealists and dreamers who are true believers and who think of what is possible. Otherwise, why would one look at children in second grade and think one of those children could win the Nobel Prize or be president? Maybe, that is why education as a discipline is so prone to fads and the constant quest for the “Holy Grail” or the next, next magic bullet. There is no one answer, there is what works for a particular population of kids.

Joy Resmovits reports at Huffington Post that the meaning of international test comparisons do not provide an accurate picture.

In International Test Scores Often Misinterpreted To Detriment Of U.S. Students, Argues New EPI Study, Resmovits reports:

Lawmakers should be more careful when using international test scores to drive education policy, argues a pair of researchers in a new paper for the left-leaning think tank Economic Policy Institute — because the results aren’t always what they appear to be.

According to a new paper released Wednesday, the average scores on international tests — the numbers over which advocates and politicians do much public hand-wringing — don’t tell the whole story of America’s academic performance, and inferences based on those averages can be misleading, Stanford education professor Martin Carnoy and researcher Richard Rothstein argue. They found that contrary to popular belief, international testing information shows that America’s low-income students have been improving over time…

Rothstein found that the U.S. is more unequal in social background, so he wondered whether differences between the average U.S. scores and those of its competitors were driven by that inequality. Rothstein said he was not surprised by his findings, given that the achievement gap between rich and poor U.S. students has always been large. “Higher social class students have higher average scores than lower social class students,” he said.

Here is a portion of the executive summary:

What do international tests really show about U.S. student performance?

By Martin Carnoy, Stanford Graduate School of Education and EPI
and Richard Rothstein, EPI

 View PDF

Download PDF

This report, however, shows that such inferences are too glib. Comparative student performance on international tests should be interpreted with much greater care than policymakers typically give it. This care is essential for three reasons:

  • First, because academic performance differences are produced by home and community as well as school influences, there is an achievement gap between the relative average performance of students from higher and lower social classes in every industrialized nation. Thus, for a valid assessment of how well American schools perform, policymakers should compare the performance of U.S. students with that of students in other countries who have been and are being shaped by approximately similar home and community environments….

We have shown that U.S. student performance, in real terms and relative to other countries, improves considerably when we estimate average U.S. scores after adjusting for U.S. social class composition and for a lack of care in sampling disadvantaged students in particular. With these adjustments, U.S. scores would rank higher among OECD countries than commonly reported in reading—fourth best instead of 14th—and in mathematics—10th best instead of 25th.

  • Second, to be useful for policy purposes, information about student performance should include how this performance is changing over time. It is not evident what lessons policymakers should draw from a country whose student performance is higher than that in the United States, if that country’s student performance has been declining while U.S student performance has been improving…. performance of all students in such countries obscures the performance of disadvantaged students.

This caution especially pertains to conventional attention to comparisons of the United States and higher-scoring Finland. Although Finland’s average scores, and scores for the most-disadvantaged children, remain substantially higher than comparable scores in the United States, scores in the United States for disadvantaged children have been rising over time, while Finland’s scores for comparable children have been declining. American policymakers should seek to understand these trends before assuming that U.S. education practice should imitate practice in Finland.

As well, U.S. trends for disadvantaged children’s PISA achievement are much more favorable than U.S. trends for advantaged children. In both reading and math, disadvantaged children’s scores have been improving while advantaged student’s scores have been stagnant. U.S. policy discussion assumes that most of problems of the U.S. education system are concentrated in schools serving disadvantaged children. Trends in PISA scores suggest that the opposite may be the case.

  • Third, different international and domestic tests sometimes seem to show similar trends, but sometimes seem quite inconsistent. These inconsistencies call into question conclusions drawn from any single assessment, and policymakers should attempt to understand the complex causes of these inconsistencies….

In our comparisons of U.S. student performance on the PISA test with student performance in six other countries—three similar post-industrial economies (France, Germany, and the United Kingdom) and three countries whose students are “top scoring” (Canada, Finland, and Korea)—we conclude that, in reading:

  • Higher social class (Group 5) U.S. students now perform as well as comparable social class students in all six comparison countries.
  • Disadvantaged students perform better (in some cases, substantially better) than disadvantaged students in the three similar post-industrial countries, but substantially less well than disadvantaged students in the three top-scoring countries.
  • The reading achievement gap between advantaged and disadvantaged students in the United States is smaller than the gap in the three similar post-industrial countries, but larger than the gap in the top-scoring countries….

These comparisons suggest that much of the discussion in the United States that points to international test comparisons to contend that U.S. schools are “failing” should be more nuanced. Although claims about relative U.S. school failure often focus on disadvantaged students’ performance, international data show that U.S. disadvantaged student performance has improved over the past decade in both mathematics and reading compared to similar social class students in all our comparison countries except Germany. TIMSS and NAEP data also show improvement for all social class groups in mathematics during the last decade. Should we consider these improvements a failure, particularly when the scores of disadvantaged students in all comparison countries but Germany have declined in this same period?

The increased rate of poverty has profound implications if this society believes that ALL children have the right to a good basic education. Moi blogs about education issues so the reader could be perplexed sometimes because moi often writes about other things like nutrition, families, and personal responsibility issues. Why? The reader might ask? Because children will have the most success in school if they are ready to learn. Ready to learn includes proper nutrition for a healthy body and the optimum situation for children is a healthy family. Many of societies’ problems would be lessened if the goal was a healthy child in a healthy family. There is a lot of economic stress in the country now because of unemployment and underemployment. Children feel the stress of their parents and they worry about how stable their family and living situation is.

Teachers and schools have been made TOTALLY responsible for the education outcome of the children, many of whom come to school not ready to learn and who reside in families that for a variety of reasons cannot support their education. All children are capable of learning, but a one-size-fits-all approach does not serve all children well. Different populations of children will require different strategies and some children will require remedial help, early intervention, and family support to achieve their education goals.


Report from Center for American Progress report: Kids say school is too easy                                                                 

Complete College America report: The failure of remediation

Book: Inequality in America affects education outcome

What exactly are the education practices of top-performing nations?                             

Where information leads to Hope. ©                 Dr.

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART©                   

Dr. Wilda Reviews ©                                     

Dr. Wilda ©                                                                          

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: