Tag Archives: Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment

Study: Teacher merit pay works in some situations

27 Jul

Teacher compensation is a hot education topic. The role of evaluations in compensation, merit pay, pay based upon credentials and higher pay for specialty areas are all hot topics and hot button issues. The Center for American Progress has a report by Frank Adamson and Linda Darling Hammond. In the report, Speaking of Salaries: What It Will Take to Get Qualified, Effective Teachers In All Communities  Adamson and Darling- Hammond write:

As Education Trust President Kati Haycock has noted, the usual statistics about teacher credentials, as shocking as they are, actually understate the degree of the problem in the most impacted schools:

The fact that only 25% of the teachers in a school are uncertified doesn’t mean that the other 75% are fine. More often, they are either brand new, assigned to teach out of field, or low-performers on the licensure exam … there are, in other words, significant numbers of schools that are essentially dumping grounds for unqualified teachers – just as they are dumping grounds for the children they serve….

Download this report (pdf)

Download the executive summary (pdf)

Melanie Smollin has an excellent post at Take Part, Five Reasons Why Teacher Turnover Is On The Rise Marguerite Roza and Sarah Yatsko from the University of Washington’s Centeron Reinventing Education have an interesting February 2010 policy brief.

In Beyond Teacher Reassignments: Better Ways School Districts Can Remedy Salary Inequities Across Schools Districts Roza and Yatsko report:

This brief addresses this concern by demonstrating that districts would NOT need to mandatorily reassign teachers. It shows that there are other ways to restructure allocations that do not systematically shortchange the neediest schools. Discussed here are four options that districts could pursue to remedy school spending inequities created by uneven salaries:

  • Option 1: Apply teacher salary bonuses to some schools to balance salaries

  • Option 2: Vary class size across schools to level spending

  • Option 3: Concentrate specialist and support staff in schools with lower-salaried teachers

  • Option 4: Equalize per-pupil dollar allocations

Download Full Report (PDF: 736 K)

Of all the issues about teacher compensation, one of the hottest is “merit pay.”

Dylan Matthews writes in the Washington Post article, Does teacher merit pay work? A new study says yes:

There’s very good evidence that teacher quality matters a lot in terms of student performance in school and success later on in life. The economist Raj Chetty of Harvard, for example, has found that students randomly placed with more experienced kindergarten teachers not only perform better on tests but earn more and save more for retirement as adults, are likelier to go to college, and go to better colleges than their peers with less experienced teachers. Eric Hanushek of Stanford estimates that a good teacher – defined as at the 84th percentile, or one standard deviation above the mean for you stats nerds – provides students with test scores associated with an increase of between $22,000 and $46,000 in lifetime earnings.

Findings like these lead some to favor “merit pay” regimes that include student test scores as a determinant of teachers’ salaries. This has met opposition from teachers’ unions and testing skeptics, who argue that it would result in teaching-to-the-test at the expense of actual learning. For a long time, the data has been mixed on merit pay. Two studies from Mathematica Policy Research in 2010 that found little benefit, while a study in Nashville found mild benefits for fifth graders but none for other students.

That has changed with the publication of a new paper (pdf) by Harvard’s Roland Fryer, the University of Chicago’s Steven Levitt (of Freakonomics fame) and John List, and UC San Diego’s Sally Sadoff. The authors went into nine K-8 schools in Chicago Heights, a city 30 miles south of Chicago, and randomly selected teachers (who had to consent, which 93.75 percent did) to take part in a merit pay scheme. The students affected were overwhelmingly low-income, with 98 percent receiving free or subsidized lunches. Teachers in the experiment were offered $80 per percentile improvement in student test scores, for a maximum reward of $8,000, compared to a typical teacher salary of $50,000.

The authors split teachers in the study into a control group, who were not offered any rewards, a “gain” group, which was promised rewards of up to $8,000 at the end of the school year, and a “loss” group, which was given $4,000 upfront and asked to pay back any rewards they did not earn. The idea behind the latter group was that loss aversion should motivate teachers to perform better than they would if they only stood to gain more money. Additionally, the gain and loss groups were split, with a “team” group being rewarded on the basis of theirs and fellow teachers’ test scores, and the “individual” group being reward only on the basis of their own scores. The conclusion: it worked, and it worked almost twice as well when the money was given at the start and then taken away…. http://www.washingtonpost.com/blogs/ezra-klein/wp/2012/07/23/does-teacher-merit-pay-work-a-new-study-says-yes/

One might ask why “merit pay’ seemed to work in the situation studied?

Jordan Weissmann writes a provocative analysis of the study in the Atlantic article, A Very Mean (But Maybe Brilliant) Way to Pay Teachers:

But Levitt, Fryer and Co. argue that there’s a serious problem with merit pay. So far, they say, there’s been scant evidence that it actually works. Studies of teacher incentive programs in Tennessee and New York City failed to find any signs that they improved student learning. In the New York experiment, which Harvard’s Fryer conducted, the impact may have even been detrimental. 

Enter loss aversion. The authors theorized that instead of offering a lump-sum bonus to teachers come summertime, it might be more effective to give instructors money upfront, then warn them that they would have to pay it back if their students didn’t hit the proper benchmarks. Rather than tap into teachers’ ambition, they’d tap into their anxiety.

To test their idea, the authors designed an experiment for the 2010-2011 school year involving 150 K-8 teachers from Chicago Heights, a low-income community in Illinois. The instructors were randomly assigned to a control group or one of two main bunches, which I’ll shorthand as the “winners” and the “losers.” The winners agreed to work under a traditional year-end bonus structure, where they could make up to $8,000 extra based on their students’ standardized test scores. The losers were given $4,000 off the bat and informed that if their students’ turned in below-average results, they’d have to pay a portion of it back commensurate with just how poor their scores were. On the flip side, an above-average performance could earn them additional bonus money, up to the full $8,000. 

The authors then divided the winners and losers again so that some teachers would be rewarded based on their results as a group, and others would be rewarded based on their results as individuals. 

Come vacation time, the losers had won. In math, paying teachers a year-end bonus had no statistically significant effect. When teachers had money to lose, though, their students over performed. The impact was large — the equivalent of improving a teacher’s skills by one full standard deviation — and the pattern held whether teachers were compensated as a group or as individuals. The authors’ data on reading scores turned out to be shakier, since most students ultimately had more than one instructor working with them on language skills, but it indicated a similar trend. 

In short, they found that merit pay can work. You just have to be tricky, and a little bit mean, with how you implement it…. http://www.theatlantic.com/business/archive/2012/07/a-very-mean-but-maybe-brilliant-way-to-pay-teachers/260234/#.UBHCJts3U6I.email

Citation:

Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment*

Roland G. Fryer, Jr.

Harvard University

Steven D. Levitt

The University of Chicago

John List

The University of Chicago

Sally Sadoff

University of California San Diego

Abstract

Domestic attempts to use financial incentives for teachers to increase student achievement have been ineffective. In this paper, we demonstrate that exploiting the power of loss aversion—teachers are paid in advance and asked to give back the money if their students do not improve sufficiently—increases math test scores between 0.201 (0.076) and 0.398 (0.129) standard deviations. This is equivalent to increasing teacher

quality by more than one standard deviation. A second treatment arm, identical to the loss aversion treatment but implemented in the standard fashion, yields smaller and statistically insignificant results. This suggests it is loss aversion, rather than other features of the design or population sampled, that leads to the stark differences between our findings and past research. 

What the various studies seem to point out is there is no one remedy which works in all situations and that there must be a menu of education options.

Resources:

A Lively Debate Over Teacher Salaries                         http://www.nytimes.com/schoolbook/2012/01/05/a-lively-debate-over-teacher-salaries/

Are Teachers Overpaid?                                                http://www.nytimes.com/roomfordebate/2012/01/02/are-teachers-overpaid/

Some Teachers Skeptical of Merit Pay                   http://www.nytimes.com/schoolbook/2012/01/13/some-teachers-skeptical-of-merit-pay/

Related:

Washington D.C. rolls out merit pay                  https://drwilda.wordpress.com/2012/01/02/washington-d-c-rolls-out-merit-pay/

Report from The Compensation Technical Working Group: Teacher compensation in Washington                   https://drwilda.wordpress.com/tag/teacher-recruitment/

Dr. Wilda says this about that ©