Tag Archives: Harvard-Smithsonian Center for Astrophysics

Mathematica study: Moving top teachers to struggling schools produces gains

10 Nov

Anne Lowrey wrote in the 2012 New York Times article, Big Study Links Good Teachers to Lasting Gain:

WASHINGTON — Elementary- and middle-school teachers who help raise their students’ standardized-test scores seem to have a wide-ranging, lasting positive effect on those students’ lives beyond academics, including lower teenage-pregnancy rates and greater college matriculation and adult earnings, according to a new study that tracked 2.5 million students over 20 years.
A study found profound effects on students whose teachers helped them raise their test scores.
The paper, by Raj Chetty and John N. Friedman of Harvard and Jonah E. Rockoff of Columbia, all economists, examines a larger number of students over a longer period of time with more in-depth data than many earlier studies, allowing for a deeper look at how much the quality of individual teachers matters over the long term.
“That test scores help you get more education, and that more education has an earnings effect — that makes sense to a lot of people,” said Robert H. Meyer, director of the Value-Added Research Center at the University of Wisconsin-Madison, which studies teacher measurement but was not involved in this study. “This study skips the stages, and shows differences in teachers mean differences in earnings.”
The study, which the economics professors have presented to colleagues in more than a dozen seminars over the past year and plan to submit to a journal, is the largest look yet at the controversial “value-added ratings,” which measure the impact individual teachers have on student test scores. It is likely to influence the roiling national debates about the importance of quality teachers and how best to measure that quality


Executive Summary of National Bureau of Economic Research Working Paper No. 17699, December 2011
Raj Chetty, Harvard University and NBER
John N. Friedman, Harvard University and NBER
Jonah E. Rockoff, Columbia University and NBER
Executive Summary http://obs.rc.fas.harvard.edu/chetty/va_exec_summ.pdf
Manuscript (NBER WP17699)
Presentation Slides http://obs.rc.fas.harvard.edu/chetty/va_exec_summ.pdf

Stephen Sawchuk reported in the Education Week article, Moving Top Teachers to Struggling Schools Has Benefits:

The transfer of top elementary teachers to low-achieving schools can help boost students’ performance, but there’s a catch: getting them to agree to move.
A new study, seven years in the making, finds that elementary teachers identified as effective who transferred to low-achieving schools under a bonus-pay program helped their new students learn more, on average, than teachers in a control group did with their students. They also stayed in the schools at least as long as other new hires.
But despite a large financial reward, only 5 percent of eligible teachers made the shift, the report concludes.
“It’s a hard sell, even with $20,000 on the table,” said Steven M. Glazerman, a senior fellow at the Princeton, N.J.-based Mathematica Policy Research, the evaluation firm that conducted the study.
Education advocates have long deplored inequitable access by disadvantaged students to high-quality teaching. The federally financed study suggests there is promise in incentive programs, but highlights the logistical complexities in carrying them out, said Sarah Almy, the director of teacher quality for the Education Trust, a Washington-based group that advocates for poor and minority students.
“I think it’s a reminder of how much we still have to understand about this issue, and that it is challenging,” she said….

Here is the press release from Mathematica:

New Study: Teachers with High “Value Added” Can Boost Test Scores in Low-Performing Schools
Known to participants as the Talent Transfer Initiative, this $20,000 financial incentive for high-performing teachers to transfer to low-performing schools had positive impacts on math and reading scores.
Contact: Jennifer de Vallance, (202) 484-4692
WASHINGTON, DC—November 6, 2013. A $20,000 incentive for high-performing teachers to move to low-performing schools has helped raise the math and reading test scores of elementary students by 4 to 10 percentile points, according to a new studyconducted by Mathematica Policy Research and sponsored by the U.S. Department of Education, Institute of Education Sciences (IES). Although there was no evidence of impacts in middle schools, the combined impact on elementary and middle school grade teams was positive and significant for reading by the second year after the transfer. The average cost of producing these gains in elementary and middle schools is estimated to be $7,000 cheaper than reducing class size to produce a similar effect. In elementary schools, the intervention was $13,000 cheaper per grade than the class-size reduction benchmark.
Many education policy experts have raised concerns that disadvantaged students do not have the same access to highly effective teachers as other students. To address this issue, IES sponsored an evaluation of an intervention known to study participants as the Talent Transfer Initiative (TTI). TTI offered a financial incentive to the teachers with the highest scores year after year on value-added measures (estimates of their ability to raise test scores, after accounting for differences between students) if they would transfer to a low-achieving school in the same district and remain there for at least two years.
About 22 percent of the selected teachers applied for the transfer, and 5 percent (81 teachers) ultimately transferred. These teachers filled 88 percent of the targeted teaching vacancies in low-performing schools. The results of the study include the following:
• TTI increased test scores in elementary schools, but not in middle schools. In classrooms targeted by TTI, impacts on elementary math and reading scores ranged from 10 to 25 percent of a standard deviation, depending on the subject and year. This is equivalent to increases of 4 to 10 percentile points. Impacts on elementary grade teams as a whole were positive in the second year, equal to 8 and 7 percent of a standard deviation in math and reading, respectively—or about 3 percentile points. Although there was no evidence of impacts in middle schools, the combined impact on elementary and middle school grade teams was positive and significant for reading by the second year after the transfer. Different outcomes in elementary versus middle schools could in part reflect differences between districts, which varied considerably in terms of impacts and where they offered TTI (elementary or middle schools).
• Most TTI teachers stayed on the job even after payments ended. TTI had a positive impact on teacher-retention rates during the first two years, while transfer teachers were receiving bonus payments. Ninety-three percent of TTI teachers remained in their positions during that period, versus 70 percent of traditionally hired teachers. Moreover, most (60 percent) of the teachers in the TTI group also continued to teach in the low-performing schools in their third year, after the payments ended.
• Compared with similar interventions, TTI was more cost-effective. The largest impacts were in elementary schools, where the cost savings could be as large as $13,000 per grade at a given school, compared with other interventions that can be equally effective in raising test scores, such as reducing class size. Including middle schools, where achievement impacts were not significant, and assuming that the total impacts persist into a third year, the estimated cost savings exceeded $40,000 per grade.
Steven Glazerman, study director and senior fellow at Mathematica, said, “For TTI to show positive effects, two things had to be true: first, the value-added measure had to contain some meaningful signal about future performance, and second, the teachers’ skills had to transfer to the teachers’ new settings. The fact that some teachers had positive impacts and then stayed beyond their two-year commitment suggests that selective transfer incentives may have lasting effects.”
About the Study: Sponsored by IES, this multisite randomized experiment was used to study the TTI intervention in 10 large, economically diverse school districts across seven states. The districts identified schools with the lowest test scores and singled out grade-subject teams with teaching vacancies. The researchers randomly assigned each team to either a treatment group, where the principals could interview and hire a TTI-transfer candidate eligible for $20,000, or to a control group, where the school principals filled the vacancies however they normally would. Researchers then followed the students and teachers in both the treatment and control groups for two years and compared their outcomes.
View the video, fact sheet, or full report on this study.
About Mathematica: Mathematica Policy Research, a nonpartisan research firm, provides a full range of research and data collection services, including program evaluation and policy research, survey design and data collection, research assessment and interpretation, and program performance/data management, to improve public well-being. Its clients include federal and state governments, foundations, and private-sector and international organizations. The employee-owned company, with offices in Princeton, N.J.; Ann Arbor, Mich.; Cambridge, Mass.; Chicago, Ill.; Oakland, Calif.; and Washington, D.C., has conducted some of the most important studies of education, disability, health care, family support, employment, nutrition, and early childhood policies and programs.

Every child has a right to a good basic education. In order to ensure that every child has a good basic education, there must be a quality teacher in every classroom.


Studies: For struggling math students, teacher quality matters https://drwilda.com/2013/04/14/studies-for-struggling-math-students-teacher-quality-matters/

The attempt to evaluate teacher colleges is getting nasty https://drwilda.wordpress.com/2012/02/05/523/

Could newest teaching strategy be made in Japan? https://drwilda.wordpress.com/2012/01/11/could-newest-teaching-strategy-be-made-in-japan/

New Harvard study about impact of teachers https://drwilda.wordpress.com/2012/01/08/new-harvard-study-about-impact-of-teachers/

Is it true that the dumbest become teachers? https://drwilda.wordpress.com/2011/12/09/is-it-true-that-the-dumbest-become-teachers/

Where information leads to Hope. © Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:


Dr. Wilda Reviews ©

Dr. Wilda ©

Harvard-Smithsonian study: The subject matter knowledge of science teachers is important element of student science learning

5 May

Moi has written about teacher quality in Studies: For struggling math students, teacher quality matters https://drwilda.com/2013/04/14/studies-for-struggling-math-students-teacher-quality-matters/

and New Harvard study about impact of teachers:

The Guide to Teacher Quality lists several key attributes of a quality teacher:


Experience is very important. The ability of a new teacher to support student learning

increases greatly during his/her first year of teaching and continues to grow through at least the

first several years of teaching (Clotfelter, Ladd & Vigdor, 2007; Clotfelter, Ladd & Vigdor, 2004;

Hanushek et al., 1998).

Teacher attrition matters. Districts and schools with relatively high rates of teacher

attrition are likely to have more inexperienced teachers and, as a result, instructional quality

and student learning suffer (Alliance for Quality Teaching, 2008).

Ability matters. Teachers with higher scores on college admission or licensure tests as well

as those from colleges with more selective admission practices are better able to support student

learning (Gitomer, 2007; Rice, 2003; Wayne and Youngs, 2003; Reichardt, 2001; Ferguson

& Ladd, 1996; Greenwald, Hedges & Laine, 1996).

Teachers’ subject matter knowledge helps students learn. Students learn when their

teacher knows the subject, particularly in secondary science and mathematics (Floden &

Meniketti, 2006; Rice, 2003; Wayne and Youngs, 2003; Reichardt, 2001).

Preparation and training in how to teach makes a difference. Knowing how to teach

improves student learning, particularly when a teacher is in his/her first years of teaching (Rice,

2003; Allen, 2003; Boyd, Grossman, Lankford, Loeb & Wyckoff, 2005).

Teacher diversity may also be important. There is emerging evidence that students learn

better from teachers of similar racial and ethnic background (Dee, 2004; Dee, 2001; Hanushek

et al. 1998).

One of the important attributes is the subject matter knowledge of the teacher. These findings are particularly important in light of the study, The Long-Term Impacts of Teachers: TeacherValue-Added and Student Outcomes in Adulthood by Raj Chetty, Harvard University and NBER , John N. Friedman, Harvard University and NBER, and Jonah E. Rockoff, Columbia University and NBER .

Here is a portion of the executive summary:

Many policy makers advocate increasing the quality of teaching, but there is considerable debate about the best way to measure and improve teacher quality. One method is to evaluate teachers based on their impacts on students’ test scores, commonly termed the “value-added” (VA) approach. A teacher’s value-added is defined as the average test-score gain for his or her students, adjusted for differences across classrooms in student characteristics such as prior scores. School districts from Washington D.C. to Los Angeles have begun to use VA to evaluate teachers. Proponents argue that using VA can improve student achievement (e.g. Hanushek 2009), while critics argue that test score gains are poor proxies for a teacher’s true quality (e.g. Baker et al. 2010).

The debate about VA stems from two fundamental questions. First, does VA accurately measure teachers’ impacts on scores or does it unfairly penalize teachers who may systematically be assigned lower achieving students? Second, do high VA teachers improve their students’ long-term outcomes or are they simply better at teaching to the test? Researchers have not reached a consensus about the accuracy and long-term impacts of VA because of data and methodological limitations.

We address these two questions by tracking one million children from a large urban school district from 4th grade to adulthood. We evaluate the accuracy of standard VA measures using several methods, including natural experiments that arise from changes in teaching staff. We find that when a high VA teacher joins a school, test scores rise immediately in the grade taught by that teacher; when a high VA teacher leaves, test scores fall. Test scores change only in the subject taught by that teacher, and the size of the change in scores matches what we predict based on the teacher’s VA. These results establish that VA accurately captures teachers’ impacts on students’ academic achievement and thereby reconcile the conflicting conclusions of Kane and Staiger (2008) and Rothstein (2010). These methods provide a simple yet powerful method to estimate the bias of value-added models in any district; interested readers can download computer code to implement these tests from this link.

In the second part of our study, we analyze whether high VA teachers also improve students’ long-term outcomes. We find that students assigned to higher VA teachers are more successful in many dimensions. They are more likely to attend college, earn higher salaries, live in better neighborhoods, and save more for retirement. They are also less likely to have children as teenagers.

Teachers’ impacts on students are substantial. Replacing a teacher whose true VA is in the bottom 5% with a teacher of average quality would generate lifetime earnings gains worth more than $250,000 for the average classroom. VA estimates are less reliable when they are based on data from a small number of classes. However, even after observing teachers’ impacts on test scores for one year, estimates of VA are reliable enough that such personnel changes would yield large gains on average.

Teachers have large impacts in all the grades we analyze (4 to 8), implying that the returns to education remain large well beyond early childhood. Teachers’ impacts on earnings are also similar in percentage terms for students from low and high income families. As a rough guideline, parents should be willing to pay about 25% of their child’s income at age 28 to switch their child from a below-average (25th percentile) to an above-average (75th percentile) teacher. For example, parents whose children will earn around $40,000 in their late 20s should be willing to pay $10,000 to switch from a below-average to an above-average teacher for one grade, based on the expected increase in their child’s lifetime earnings.

Overall, our study shows that great teachers create great value – perhaps several times their annual salaries – and that test score impacts are helpful in identifying such teachers. However, more work is needed to determine the best way to use VA for policy. For example, using VA in teacher evaluations could induce undesirable responses that make VA a poorer measure of teacher quality, such as teaching to the test or cheating. There will be much to learn about these issues from school districts that start using VA to evaluate teachers. Nevertheless, it is clear that improving the quality of teaching – whether using value-added or other tools – is likely to have large economic and social returns.

See, Annie Lowrey’s New York Times article, Big Study Links Good Teachers to Lasting Gain

Erik Robelen writes in the Education Week article, Knowing Student Misconceptions Key to Science Teaching, Study Finds which finds that teacher knowledge matters in science teaching.

It seems obvious that teachers need to understand the content they’re trying to convey to students. But a new study finds that what’s especially critical to improved science learning is that teachers also know the common misconceptions students have. And in science, there are plenty of things that young people—and a lot of adults—don’t correctly understand, such as what causes the change of seasons.

The study, conducted by researchers at the Harvard-Smithsonian Center for Astrophysics, targeted middle school physical science. The researchers enlisted 181 teachers to administer a multiple-choice test of student knowledge of science concepts. Twelve of the 20 items were designed to have a “particularly wrong answer corresponding to a commonly held misconception,” explained Philip Sadler, the lead author and a senior lecturer at the Harvard-Smithsonian center.

The “unusual” part of the study, he said, was that teachers also took the test, and were asked to identify both the correct answer and the one students were most often likely to incorrectly select. Although the teachers overall did “quite well” at selecting the correct answer, the results were more mixed in predicting students’ incorrect response.

“Teacher knowledge was predictive of higher student gains. No surprise there,” Sadler explained in an email. “However, for more difficult concepts where many students had a misconception, only teachers who knew the science and the common misconceptions have large student gains.” What’s key, he said, is knowing “what was going on in their students’ heads.”

The study, supported by funding from the National Science Foundation, was recently published online in the American Educational Research Journal. The study also is the focus of an article published yesterday in Science Daily.

The researchers acknowledge that many educators question the value of tests composed of multiple-choice items, but said in the study that when items are written to include popular misconceptions as “distractors, they function well in diagnosing misconceptions that impede the learning of science.” The test questions were based on concepts covered in a set of science content standards published by the National Research Council in 1996. Topics addressed included properties of matter, motions and forces, and transfer of energy.

In his email, Sadler said the study was sparked by the reaction many educators have had to a video he helped to develop, called “A Private Universe,” in which graduating Harvard University seniors reveal “the same wrong ideas” about science as middle-school students. http://blogs.edweek.org/edweek/curriculum/2013/05/it_goes_without_saying_that.html


The Influence of Teachers’ Knowledge on Student Learning in Middle School Physical Science Classrooms

  1. Philip M. Sadler,

  2. Gerhard Sonnert,

  3. Harold P. Coyle,

  4. Nancy Cook-Smith and

  5. Jaimie L. Miller

+ Author Affiliations

  1. Harvard-Smithsonian Center for Astrophysics


This study examines the relationship between teacher knowledge and student learning for 9,556 students of 181 middle school physical science teachers. Assessment instruments based on the National Science Education Standards with 20 items in common were administered several times during the school year to both students and their teachers. For items that had a very popular wrong answer, the teachers who could identify this misconception had larger classroom gains, much larger than if the teachers knew only the correct answer. On items on which students did not exhibit misconceptions, teacher subject matter knowledge alone accounted for higher student gains. This finding suggests that a teacher’s ability to identify students’ most common wrong answer on multiple-choice items, a form of pedagogical content knowledge, is an additional measure of science teacher competence.

This Article

  1. Published online before print March 6, 2013, doi: 10.3102/0002831213477680 Am Educ Res J March 6, 2013 0002831213477680
  1. » AbstractFree

  2. Full Text

  3. Full Text (PDF)

Every child has a right to a good basic education. In order to ensure that every child has a good basic education, there must be a quality teacher in every classroom.


National Council on Teacher Quality

How I was evaluated as a first-year teacher – 10/04/2012

A first year teacher, ripe for feedback and improvement, gets none.

What’s the Latest on Teacher Evaluation? – 10/02/2012

We’ve got the skinny on what states are doing to evaluate their teachers and award them tenure. Read more

2011 State Teacher Policy Yearbook

January 2012

State of the States: Trends and Early Lessons on Teacher Evaluation and Effectiveness Policies

October 2011

State of the States 2012: Teacher Effectiveness Policies

October 2012

Center for Teaching Qualityhttp://www.teachingquality.org/

The National Comprehensive Center for Teacher Quality http://www.tqsource.org/


Linda Darling-Hammond on teacher evaluation https://drwilda.wordpress.com/2012/05/09/linda-darling-hammond-on-teacher-evaluation/

Report: Measuring teacher effectiveness                   https://drwilda.wordpress.com/2012/06/13/report-measuring-teacher-effectiveness/

Where information leads to Hope. ©                               Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART©                      http://drwildaoldfart.wordpress.com/

Dr. Wilda Reviews ©                                             http://drwildareviews.wordpress.com/

Dr. Wilda ©                                                                                                    https://drwilda.com/