University of Illinois Urbana-Champaign study: Many education experts lack expertise

23 Feb

Reporters, seminar sponsors, political town halls often rely on “experts” to give an opinion about a particular issue. Education is an area populated by “experts” from all over the pedagogical and political spectrum. What exactly is an expert? The Legal Information Institute provides the legal definition of expert:

Rule 702. Testimony by Expert Witnesses
A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if:
(a) the expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue;
(b) the testimony is based on sufficient facts or data;
(c) the testimony is the product of reliable principles and methods; and
(d) the expert has reliably applied the principles and methods to the facts of the case.
Notes (Pub. L. 93–595, §1, Jan. 2, 1975, 88 Stat. 1937; Apr. 17, 2000, eff. Dec. 1, 2000; Apr. 26, 2011, eff. Dec. 1, 2011.) https://www.law.cornell.edu/rules/fre/rule_702

The legal definition provides some guidance.

Science Daily reported in the article, Education ‘experts’ cited in news stories may lack expertise, study finds:

The people most often cited as “education experts” in blogs and news stories may have the backing of influential organizations — but have little background in education and education policy, a new study suggests.

The findings are cause for concern because some prominent interest groups are promoting reform agendas and striving to influence policymakers and public opinion using individuals who have substantial media relations skills but little or no expertise in education research, say the authors of the study, Joel R. Malin and Christopher Lubienski, both at the University of Illinois.

To examine possible links between individuals’ media presence and their levels of expertise, Malin and Lubienski compiled a diverse list of nearly 300 people who appeared on the lists of experts prepared by several major education advocacy and policy organizations, including the conservative American Enterprise Institute and the liberal National Education Policy Center.
Malin and Lubienski also added to their sample a handful of scholars not on those lists but who are prominent and influential in the field of education.

Each person’s level of expertise was then scored using a formula that included their number of Google Scholar citations; their years of experience, calculated by subtracting the year they attained their highest degree from 2014; and whether or not the person had earned a doctoral or equivalent degree.

Each person’s level of media influence was calculated based upon the number of times they were quoted or mentioned in education press, U.S. newspapers or blogs during 2013; whether they had a Twitter profile; and their Klout score, which is a proxy for social media influence.
Experts were more likely to be quoted or mentioned in newspapers and blogs if they had higher scores on Google Scholar, Malin and Lubienski found. Every 1-point increase in an expert’s Google Scholar score was associated with a 1-percent increase in blog mentions.
Accordingly, each 1-point increase in years of experience corresponded with an increase of about 1 percent in newspaper citations, the researchers found.

However, affiliation with a policy or advocacy organization also substantially increased an expert’s media presence. People associated with the American Enterprise Institute were nearly 2.5 times more likely to be cited in education media.

Likewise, experts were 1.78 and 1.5 times more likely to be mentioned in blogs if they were affiliated with Cato or the American Enterprise Institute, respectively.

Although the initial list included 287 experts, Malin and Lubienski could not find the necessary information to estimate 52 of these individuals’ years of experience. More than half of these people were connected to organizations such as Cato and Heritage.

While the three people in the sample who were affiliated with Cato each received the maximum number of points for blog mentions, these individuals’ average estimated expertise score was 4.67 — substantially lower than the average score for the full sample, which was greater than 20. Perhaps the most troubling finding was that possession of a doctoral degree was associated with 67 percent fewer blog citations and 60 percent fewer newspaper mentions, and fewer Klout points, which indicates that academic researchers with empirical expertise in education are often far removed from popular and policy conversations, Malin and Lubienski said…. http://www.sciencedaily.com/releases/2015/02/150220133338.htm

Citation:

Education ‘experts’ cited in news stories may lack expertise, study finds
Date: February 20, 2015

Source: University of Illinois at Urbana-Champaign
Summary:
A study of education experts cited in news stories and blogs during 2013 finds that some lack background in education policy and research.
Educational Expertise, Advocacy, and Media Influence
Joel R. Malin, Christopher Lubienski

Abstract
The efforts of many advocacy organizations to advance their preferred policies despite conflicting evidence of the effectiveness of these policies raise questions about factors that shape successful policy promotion. While many may like to think that expertise on an issue in question is an essential prerequisite for influence in public policy discussions, there is a traditional disconnect between research evidence and policymaking in many fields, including education. Moreover, the efforts of many policy advocates suggest that they see advantages in other factors besides research expertise in advancing their interpretation of evidence for use in policymaking processes. We hypothesize that some of the most influential education-focused organizations are advancing their agendas by engaging media and drawing on individuals who possess substantial media acumen, yet may not possess traditionally defined educational expertise. Thus, we hypothesize that media impact is loosely coupled with educational expertise. In fact, in analyzing various indicators of expertise and media penetration, we find a weak relationship between expertise and media impact, but find significantly elevated media penetration for individuals working at a sub-sample of organizations promoting what we term “incentivist” education reforms, in spite of their generally lower levels of expertise. We find these organizations are particularly effective in engaging new media forms by going directly to their audience. We consider the policy implications in the concluding discussion.
Keywords
agenda setting; decision making; educational policy; expertise; information dissemination; political influences; politics
Full Text:
PDF http://epaa.asu.edu/ojs/article/view/1706/1456
Related Articles
Jabbar, H., Goldie, D., Linick, M., & Lubienski, C. (2014). Using bibliometric and social media analyses to explore the “echo chamber” hypothesis. Educational Policy, 28(2), 281-305. http://dx.doi.org/10.1177/0895904813515330

Lubienski, C., Scott, J., & DeBray, E. (2014). The politics of research use in education policymaking. Educational Policy, 28(2), 131-144. http://dx.doi.org/10.1177/0895904813515329

Henig, J. (2009). Politicization of evidence: Lessons for an informed democracy. Educational Policy, 23, 137-160. http://dx.doi.org/10.1177/0895904808328525

DOI: http://dx.doi.org/10.14507/epaa.v23.1706

Here is the press release from the University of Illinois:

Education ‘experts’ may lack expertise, study finds
2/20/2015 | Sharita Forrest, Education and Social Work Editor | 217-244-1072; slforres@illinois.edu

CHAMPAIGN, Ill. — The people most often cited as “education experts” in blogs and news stories may have the backing of influential organizations – but have little background in education and education policy, a new study suggests.

The findings are cause for concern because some prominent interest groups are promoting reform agendas and striving to influence policymakers and public opinion using individuals who have substantial media relations skills but little or no expertise in education research, say the authors of the study, Joel R. Malin and Christopher Lubienski, both at the University of Illinois.

To examine possible links between individuals’ media presence and their levels of expertise, Malin and Lubienski compiled a diverse list of nearly 300 people who appeared on the lists of experts prepared by several major education advocacy and policy organizations, including the conservative American Enterprise Institute and the liberal National Education Policy Center.

Malin and Lubienski also added to their sample a handful of scholars not on those lists but who are prominent and influential in the field of education.
Each person’s level of expertise was then scored using a formula that included their number of Google Scholar citations; their years of experience, calculated by subtracting the year they attained their highest degree from 2014; and whether or not the person had earned a doctoral or equivalent degree.
Each person’s level of media influence was calculated based upon the number of times they were quoted or mentioned in education press, U.S. newspapers or blogs during 2013; whether they had a Twitter profile; and their Klout score, which is a proxy for social media influence.

Experts were more likely to be quoted or mentioned in newspapers and blogs if they had higher scores on Google Scholar, Malin and Lubienski found. Every 1-point increase in an expert’s Google Scholar score was associated with a 1-percent increase in blog mentions.

Accordingly, each 1-point increase in years of experience corresponded with an increase of about 1 percent in newspaper citations, the researchers found.
However, affiliation with a policy or advocacy organization also substantially increased an expert’s media presence. People associated with the American Enterprise Institute were nearly 2.5 times more likely to be cited in education media.

Likewise, experts were 1.78 and 1.5 times more likely to be mentioned in blogs if they were affiliated with Cato or the American Enterprise Institute, respectively.
Although the initial list included 287 experts, Malin and Lubienski could not find the necessary information to estimate 52 of these individuals’ years of experience. More than half of these people were connected to organizations such as Cato and Heritage.

While the three people in the sample who were affiliated with Cato each received the maximum number of points for blog mentions, these individuals’ average estimated expertise score was 4.67 – substantially lower than the average score for the full sample, which was greater than 20.
Perhaps the most troubling finding was that possession of a doctoral degree was associated with 67 percent fewer blog citations and 60 percent fewer newspaper mentions, and fewer Klout points, which indicates that academic researchers with empirical expertise in education are often far removed from popular and policy conversations, Malin and Lubienski said.

“Our findings suggest that individuals with less expertise can often have greater success in media penetration,” said Malin, a curriculum specialist with the Pathways Resource Center and a doctoral candidate in educational administration and leadership at the university. “Although some individuals might not have formal training in research methods for analyzing the issues about which they are speaking, they possess skills and orientations that make them accessible and appealing to the media. And when these people are affiliated with organizations that have strong media arms or outreach efforts, they have the support and the incentive to engage broader and policy audiences.“
“Newer forms of media offer particularly useful opportunities for directly engaging audiences, while bypassing traditional forms of quality checks on expertise,” said Lubienski, a professor of education policy and director of the Forum on the Future of Public Education at the university. “We believe caution and consideration of individuals’ expertise are warranted when reporters and bloggers are researching topics and seeking insights – and when policymakers and laypersons are consuming media.”

Researchers who want to see their work have impact beyond the academic community must become more adept at communicating via traditional and new media. Otherwise, policy changes in education will be guided more by ideology and agendas than by research, Malin and Lubienski said.
The study was published in a recent issue of the journal Education Policy Analysis Archives.

Editor’s note: To reach Joel Malin, call 217-244-9390; email jrmalin2@illinois.edu.
To reach Christopher Lubienski, call 217-333-4382; email club@illinois.edu

The paper, “Educational Expertise, Advocacy, and Media Influence,” is available online or from the News Bureau. http://news.illinois.edu/news/15/0220expertise_JoelMalin_ChristopherLubienski.html

The point of this study is that critical thinking cannot be stressed enough when evaluating the validity or worth of any opinion.

“Incestuous, homogeneous fiefdoms of self-proclaimed expertise are always rank-closing and mutually self-defending, above all else.”

Glenn Greenwald

Where information leads to Hope. © Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART©
http://drwildaoldfart.wordpress.com/

Dr. Wilda Reviews ©
https://drwildareviews.wordpress.com/

Dr. Wilda ©
https://drwilda.com/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: