Tag Archives: PISA

Research paper: Interpreting international test scores in light of social class differences

15 Jan

Moi wrote about international student rankings in Important Harvard report about U.S. student achievement ranking:

More and more, individuals with gravitas are opining about the American education system for reasons ranging from national security to economic competitiveness. In Condoleezza Rice and Joel Klein report about American Education, moi wrote:

The Council on Foreign Relations has issued the report, U.S. Education Reform and National Security. The chairs for the report are Joel I. Klein, News Corporation and Condoleezza Rice, Stanford University. Moi opined about the state of education in U.S. education failure: Running out of excuses https://drwilda.wordpress.com/2011/12/13/u-s-education-failure-running-out-of-excuses/ Education tends to be populated by idealists and dreamers who are true believers and who think of what is possible. Otherwise, why would one look at children in second grade and think one of those children could win the Nobel Prize or be president? Maybe, that is why education as a discipline is so prone to fads and the constant quest for the “Holy Grail” or the next, next magic bullet. There is no one answer, there is what works for a particular population of kids. https://drwilda.wordpress.com/2012/03/19/condoleezza-rice-and-joel-klein-report-about-american-education/

Joy Resmovits reports at Huffington Post that the meaning of international test comparisons do not provide an accurate picture.

In International Test Scores Often Misinterpreted To Detriment Of U.S. Students, Argues New EPI Study, Resmovits reports:

Lawmakers should be more careful when using international test scores to drive education policy, argues a pair of researchers in a new paper for the left-leaning think tank Economic Policy Institute — because the results aren’t always what they appear to be.

According to a new paper released Wednesday, the average scores on international tests — the numbers over which advocates and politicians do much public hand-wringing — don’t tell the whole story of America’s academic performance, and inferences based on those averages can be misleading, Stanford education professor Martin Carnoy and researcher Richard Rothstein argue. They found that contrary to popular belief, international testing information shows that America’s low-income students have been improving over time…

Rothstein found that the U.S. is more unequal in social background, so he wondered whether differences between the average U.S. scores and those of its competitors were driven by that inequality. Rothstein said he was not surprised by his findings, given that the achievement gap between rich and poor U.S. students has always been large. “Higher social class students have higher average scores than lower social class students,” he said. http://www.huffingtonpost.com/2013/01/15/international-test-scores_n_2479994.html?utm_hp_ref=education

Here is a portion of the executive summary:

What do international tests really show about U.S. student performance?

By Martin Carnoy, Stanford Graduate School of Education and EPI
and Richard Rothstein, EPI

 View PDF

Download PDF

This report, however, shows that such inferences are too glib. Comparative student performance on international tests should be interpreted with much greater care than policymakers typically give it. This care is essential for three reasons:

  • First, because academic performance differences are produced by home and community as well as school influences, there is an achievement gap between the relative average performance of students from higher and lower social classes in every industrialized nation. Thus, for a valid assessment of how well American schools perform, policymakers should compare the performance of U.S. students with that of students in other countries who have been and are being shaped by approximately similar home and community environments….

We have shown that U.S. student performance, in real terms and relative to other countries, improves considerably when we estimate average U.S. scores after adjusting for U.S. social class composition and for a lack of care in sampling disadvantaged students in particular. With these adjustments, U.S. scores would rank higher among OECD countries than commonly reported in reading—fourth best instead of 14th—and in mathematics—10th best instead of 25th.

  • Second, to be useful for policy purposes, information about student performance should include how this performance is changing over time. It is not evident what lessons policymakers should draw from a country whose student performance is higher than that in the United States, if that country’s student performance has been declining while U.S student performance has been improving…. performance of all students in such countries obscures the performance of disadvantaged students.

This caution especially pertains to conventional attention to comparisons of the United States and higher-scoring Finland. Although Finland’s average scores, and scores for the most-disadvantaged children, remain substantially higher than comparable scores in the United States, scores in the United States for disadvantaged children have been rising over time, while Finland’s scores for comparable children have been declining. American policymakers should seek to understand these trends before assuming that U.S. education practice should imitate practice in Finland.

As well, U.S. trends for disadvantaged children’s PISA achievement are much more favorable than U.S. trends for advantaged children. In both reading and math, disadvantaged children’s scores have been improving while advantaged student’s scores have been stagnant. U.S. policy discussion assumes that most of problems of the U.S. education system are concentrated in schools serving disadvantaged children. Trends in PISA scores suggest that the opposite may be the case.

  • Third, different international and domestic tests sometimes seem to show similar trends, but sometimes seem quite inconsistent. These inconsistencies call into question conclusions drawn from any single assessment, and policymakers should attempt to understand the complex causes of these inconsistencies….

In our comparisons of U.S. student performance on the PISA test with student performance in six other countries—three similar post-industrial economies (France, Germany, and the United Kingdom) and three countries whose students are “top scoring” (Canada, Finland, and Korea)—we conclude that, in reading:

  • Higher social class (Group 5) U.S. students now perform as well as comparable social class students in all six comparison countries.
  • Disadvantaged students perform better (in some cases, substantially better) than disadvantaged students in the three similar post-industrial countries, but substantially less well than disadvantaged students in the three top-scoring countries.
  • The reading achievement gap between advantaged and disadvantaged students in the United States is smaller than the gap in the three similar post-industrial countries, but larger than the gap in the top-scoring countries….

These comparisons suggest that much of the discussion in the United States that points to international test comparisons to contend that U.S. schools are “failing” should be more nuanced. Although claims about relative U.S. school failure often focus on disadvantaged students’ performance, international data show that U.S. disadvantaged student performance has improved over the past decade in both mathematics and reading compared to similar social class students in all our comparison countries except Germany. TIMSS and NAEP data also show improvement for all social class groups in mathematics during the last decade. Should we consider these improvements a failure, particularly when the scores of disadvantaged students in all comparison countries but Germany have declined in this same period? http://www.epi.org/publication/us-student-performance-testing/

The increased rate of poverty has profound implications if this society believes that ALL children have the right to a good basic education. Moi blogs about education issues so the reader could be perplexed sometimes because moi often writes about other things like nutrition, families, and personal responsibility issues. Why? The reader might ask? Because children will have the most success in school if they are ready to learn. Ready to learn includes proper nutrition for a healthy body and the optimum situation for children is a healthy family. Many of societies’ problems would be lessened if the goal was a healthy child in a healthy family. There is a lot of economic stress in the country now because of unemployment and underemployment. Children feel the stress of their parents and they worry about how stable their family and living situation is.

Teachers and schools have been made TOTALLY responsible for the education outcome of the children, many of whom come to school not ready to learn and who reside in families that for a variety of reasons cannot support their education. All children are capable of learning, but a one-size-fits-all approach does not serve all children well. Different populations of children will require different strategies and some children will require remedial help, early intervention, and family support to achieve their education goals.


Report from Center for American Progress report: Kids say school is too easy                                                                           https://drwilda.wordpress.com/2012/07/10/report-from-center-for-american-progress-report-kids-say-school-is-too-easy/

Complete College America report: The failure of remediation https://drwilda.wordpress.com/2012/06/21/complete-college-america-report-the-failure-of-remediation/

Book: Inequality in America affects education outcome https://drwilda.wordpress.com/2012/06/10/book-inequality-in-america-affects-education-outcome/

What exactly are the education practices of top-performing nations?                                       http://drwilda.wordpress.com/2012/05/28/what-exactly-are-the-education-practices-of-top-performing-nations/

Where information leads to Hope. ©                 Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART©                             http://drwildaoldfart.wordpress.com/

Dr. Wilda Reviews ©                                               http://drwildareviews.wordpress.com/

Dr. Wilda ©                                                                                    https://drwilda.com/

U.S. education failure: Running out of excuses

13 Dec

Education tends to be populated by idealists and dreamers who are true believers and who think of what is possible. Otherwise, why would one look at children in second grade and think one of those children could win the Nobel Prize or be president? Maybe, that is why education as a discipline is so prone to fads and the constant quest for the “Holy Grail” or the next, next magic bullet. There is no one answer, there is what works for a particular population of kids

Jay Mathews of the Washington Post is reporting in the article, U.S. school excuses challenged about a new book by Marc S. Tucker, “Surpassing Shanghai: An Agenda for American Education Built on the World’s Leading Systems.” In his book, Tucker examines some of the excuses which have been used to justify the failure of the American education system.

Here are some common excuses for poor U.S. performance and why Tucker thinks they are wrong. I also have included commentary from Brookings Institution scholar Tom Loveless, an expert on PISA.

  1. Our scores are lower because so many of our children are from immigrant families speaking different languages. Tucker says “the reading performance of children without an immigrant background in the United States is only marginally better than the performance of all students. It turns out that Canada, New Zealand, Australia and Hong Kong, all with percentages of immigrant students equal to or greater than the United States, all out-perform the United States in reading.” Loveless says Tucker needs to prove that immigrants in those countries are as poor and culturally deprived as U.S. immigrants.
  2. Our suburban kids do fine, but our national average PISA results are dragged down by urban schools that serve low-income students. In fact, Tucker says, the U.S. suburban average is only slightly above the average for all developed nations in the Organization for Economic Cooperation and Development, which sponsors PISA.
  3. If top-performing countries had to educate as many disadvantaged students as we do, they would not perform as well. PISA has results for what it calls “resilient” students, those who are in the bottom quarter of an index of economic, social and cultural status but who score in the top quarter of the PISA achievement measures. The higher portion of students like that in a country, the theory goes, the better its schools are doing in educating the students who are most difficult to teach. The percentage of resilient students in the United States is below the PISA average. Twenty-seven countries, including Mexico, are ahead of us. Loveless wonders if this says anything besides “countries that score higher than us score higher than us.”
  4. If we spent more on education, we would have better results. In fact, Tucker could find only one OECD country, Luxembourg, that spends more per pupil than we do, even though we score only average in reading and below average in math and science. The key factor, he says, is what we spend the money on. If we measure teacher compensation by how much teachers are paid compared to other professions requiring the same years of education, only three OECD countries pay their teachers less than we do.
  5. If we emphasize reducing class sizes, our students will do better. The PISA data shows otherwise. Countries that give higher priority to raising teacher salaries than reducing class sizes have better achievement rates than countries like ours that do the opposite. Loveless says he is sympathetic to this argument and the previous one, but would like to see evidence of causality.


There are certain elements that successful schools share.

The Wisconsin Department of Education has a good guide about successful schools. Chapter One, Characteristics of Successful Schools , lists key elements:

Chapter 1 describes the seven characteristics that comprise a successful school. Briefly, they are:

  • Vision: having a common understanding of goals, principles and expectations for everyone in the learning-community
  • Leadership: having a group of individuals dedicated to helping the learning-community reach its vision
  • High Academic Standards: describing what students need to know and be able to do
  • Standards of the Heart: helping all within the learning community become caring, contributing, productive, and responsible citizens
  • Family School and Community Partnerships: “making room at the table” for a child’s first and most influential teachers
  • Professional Development: providing consistent, meaningful opportunities for adults in the school setting to engage in continuous learning
  • Evidence of Success: collecting and analyzing data about students, programs, and staff

Like, unhappy families, failing schools are probably failing in their own way.

Happy families are all alike; every unhappy family is unhappy in its own way.

Leo Tolstoy, Anna Karenina, Chapter 1, first line
Russian mystic & novelist (1828 – 1910)

It seems everything old becomes new once again, although a relentless focus on the basics never went out of style.

Good Schools really are relentless about the basics.

Dr. Wilda says this about that ©