University of Maryland Baltimore County study: New method to address deep-seated biases in science

18 Oct

The New York Times reported in Hoaxers Slip Breastaurants and Dog-Park Sex Into Journals:

One paper, published in a journal called Sex Roles, said that the author had conducted a two-year study involving “thematic analysis of table dialogue” to uncover the mystery of why heterosexual men like to eat at Hooters.
Another, from a journal of feminist geography, parsed “human reactions to rape culture and queer performativity” at dog parks in Portland, Ore., while a third paper, published in a journal of feminist social work and titled “Our Struggle Is My Struggle,” simply scattered some up-to-date jargon into passages lifted from Hitler’s “Mein Kampf.”
Such offerings may or may not have raised eyebrows among the journals’ limited readerships. But this week, they unleashed a cascade of mockery — along with a torrent of debate about ethics of hoaxes, the state of peer review and the excesses of academia — when they were revealed to be part of an elaborate prank aimed squarely at what the authors labeled “grievance studies.”
“Something has gone wrong in the university — especially in certain fields within the humanities,” the three authors of the fake papers wrote in an article in the online journal Areo explaining what they had done. “Scholarship based less upon finding truth and more upon attending to social grievances has become firmly established, if not fully dominant, within these fields.”
Their project quickly drew comparisons to a famous 1996 hoax in which the physicist Alan Sokal got a paper mixing postmodern philosophy with the theory of quantum gravity into a prestigious cultural studies journal.
But while that hoax involved a single article, the new one involved 20 papers, produced every two weeks or so, submitted to various journals over nearly a year.
The authors — Helen Pluckrose, James A. Lindsay and Peter Boghossian — said that four papers had been published; three had been accepted but not yet published; seven were under review and six had been rejected…. https://www.nytimes.com/2018/10/04/arts/academic-journals-hoax.html

Scientific method is important because it directs inquiry into a rational discussion and outcome.

William Harris wrote in How the Scientific Method Works:

Importance of the Scientific Method
The scientific method attempts to minimize the influence of bias or prejudice in the experimenter. Even the best-intentioned scientists can’t escape bias. It results from personal beliefs, as well as cultural beliefs, which means any human filters information based on his or her own experience. Unfortunately, this filtering process can cause a scientist to prefer one outcome over another. For someone trying to solve a problem around the house, succumbing to these kinds of biases is not such a big deal. But in the scientific community, where results have to be reviewed and duplicated, bias must be avoided at all costs.
¬T¬hat’s the job of the scientific method. It provides an objective, standardized approach to conducting experiments and, in doing so, improves their results. By using a standardized approach in their investigations, scientists can feel confident that they will stick to the facts and limit the influence of personal, preconceived notions. Even with such a rigorous methodology in place, some scientists still make mistakes. For example, they can mistake a hypothesis for an explanation of a phenomenon without performing experiments. Or they can fail to accurately account for errors, such as measurement errors. Or they can ignore data that does not support the hypothesis…. https://science.howstuffworks.com/innovation/scientific-experiments/scientific-method9.htm

A University of Maryland Baltimore County (UMBC) examined biases in scientific research.

Science Daily reported in New method to address deep-seated biases in science:

New UMBC research is helping dismantle gender and publication biases in science. A team of researchers working across disciplines has developed a new statistical technique to understand similarity, rather than difference, in the natural world. With this new technique, they’ve determined that among Eastern Bluebirds the structure of songs female birds sing is statistically indistinguishable from songs males sing….
Challenging a paradigm
Working together, the team modified a statistical method used in generic drug testing to meet their needs for ecology and animal behavior studies. The existing test helps determine whether generic and brand name drugs are “statistically equivalent,” meaning they are similar enough to be prescribed safely for the same purpose. The new modification will allow scientists in other fields to test for equivalence. Before, researchers could only report they did not find a significant difference — a very different statement than saying two things are conclusively equivalent.
“We’re really hoping this new method is going to address some issues with what kinds of data get published,” Rose says. “The most important thing about being a good scientist is to be unbiased. And the whole tradition of testing for difference really leads to incredible biases in scientists,” Omland says. He adds, “There’s a whole realm of things in nature that we find interesting and important because of their similarity.”
For example, in addition to similarities in songs between the sexes in birds, researchers could use the new test to ask if two species use the same type of habitat, respond the same way to predators, or consume the same food sources. Answers to those questions could fill long-standing knowledge gaps, or even inform conservation efforts.
“This test is really broadly applicable,” says Rose, “and we’re hoping to introduce it more to the ecology and evolution field.”
A new approach
One advantage of the new method is it accounts for unequal sample sizes. In a medical study, researchers can carefully control the size of treatment and control groups. In other fields, from ecology, to engineering, to agriculture, that’s often not possible. The new test also allows researchers to determine the equivalence of several traits simultaneously, Mathew explains. For example, in this study, the authors found that the male and female birds’ songs were statistically equivalent across five different characteristics, such as duration of each song and the range of pitches the birds produced.
Rather than testing whether two things are exactly equal, the team was looking for a way to determine if two things were “close enough,” given a defined allowable margin of difference. Because of that added layer, “There are additional challenges here,” Mathew says…. https://www.sciencedaily.com/releases/2018/10/181016150725.htm

Citation:

New method to address deep-seated biases in science
A new statistical method that explicitly tests for equivalence, rather than difference, can help combat the bias against publishing
Date: October 16, 2018
Source: University of Maryland Baltimore County
Summary:
A new statistical method that tests for equivalence, rather than difference, has a role to play in dismantling gender and publication biases in science. The authors believe the technique has broad applicability across disciplines and can help remove publication bias against ”negative results,” opening the door to a broader investigation of natural phenomena.

Journal Reference:
Evangeline M. Rose, Thomas Mathew, Derek A. Coss, Bernard Lohr, Kevin E. Omland. A new statistical method to test equivalence: an application in male and female eastern bluebird song. Animal Behaviour, 2018; 145: 77 DOI: 10.1016/j.anbehav.2018.09.004

Here is the press release from UMBC:

New methods to fight biases in science, starting with birds
October 18, 2018 1:28 PM
New UMBC research is helping dismantle gender and publication biases in science. A team of researchers working across disciplines has developed a new statistical technique to understand similarity, rather than difference, in the natural world. With this new technique, they’ve determined that among Eastern Bluebirds the structure of songs female birds sing is statistically indistinguishable from songs males sing.
Awareness of female birdsong is growing worldwide, thanks in part to a breakthrough paper by Karan Odom, Ph.D. ’16, biological sciences, but it’s still understood as a trait found primarily in tropical birds. Evangeline Rose, a current Ph.D. student in the same lab and first author on a new paper in Animal Behavior, wanted to look at song in a temperate species.
During Rose’s fieldwork, “I was finding that the females were singing, to me, what sounded just like male songs,” she says. “So we started thinking about equality, and equivalence, and how to test for it.” On the advice of her advisor, Kevin Omland, professor of biological sciences, she reached out to Thomas Mathew, professor of statistics, who has expertise in statistical equivalence.
Challenging a paradigm
Working together, the team modified a statistical method used in generic drug testing to meet their needs for ecology and animal behavior studies. The existing test helps determine whether generic and brand name drugs are “statistically equivalent,” meaning they are similar enough to be prescribed safely for the same purpose. The new modification will allow scientists in other fields to test for equivalence. Before, researchers could only report they did not find a significant difference—a very different statement than saying two things are conclusively equivalent.
“We’re really hoping this new method is going to address some issues with what kinds of data get published,” Rose says. “The most important thing about being a good scientist is to be unbiased. And the whole tradition of testing for difference really leads to incredible biases in scientists,” Omland says. He adds, “There’s a whole realm of things in nature that we find interesting and important because of their similarity.”
For example, in addition to similarities in songs between the sexes in birds, researchers could use the new test to ask if two species use the same type of habitat, respond the same way to predators, or consume the same food sources. Answers to those questions could fill long-standing knowledge gaps, or even inform conservation efforts.
“This test is really broadly applicable,” says Rose, “and we’re hoping to introduce it more to the ecology and evolution field.”
A new approach
One advantage of the new method is it accounts for unequal sample sizes. In a medical study, researchers can carefully control the size of treatment and control groups. In other fields, from ecology, to engineering, to agriculture, that’s often not possible. The new test also allows researchers to determine the equivalence of several traits simultaneously, Mathew explains. For example, in this study, the authors found that the male and female birds’ songs were statistically equivalent across five different characteristics, such as duration of each song and the range of pitches the birds produced.
Rather than testing whether two things are exactly equal, the team was looking for a way to determine if two things were “close enough,” given a defined allowable margin of difference. Because of that added layer, “There are additional challenges here,” Mathew says.
“Even though this methodology is out there, it hasn’t been applied—even in statistics—with this kind of data. That’s why I was very excited when they brought this project to me,” Mathew says. Rose adds, “It ended up being a really great partnership to look at these questions that hadn’t been asked before for female song, and we also ended up modifying this test in a really cool, new way.”
Changing science
As research on similarities grows, there is also a growing drive to remove the bias against publishing studies that do not find a significant difference, often termed a “negative result.” This paper “is part of an amazing drumbeat that’s building up in the scientific community,” Omland says. “There’s a broader problem with the scientific method that’s being increasingly acknowledged, and the test we’ve developed can at least play a small role, and I hope a big role, in addressing it.”
Rose, who plans to next investigate the function of female bluebird songs, says she will carry these new techniques with her as she moves through her research career. “I think in the future, I’ll be thinking about how equivalence can change the questions we’re asking, and I’ll always keep in mind that we have extra tools in the toolkit.”
Image: An Eastern Bluebird, Rose’s study organism, sits on a fence. Photo by Dolan Trout, used under CC 2.0.

Honesty in scientific research protocol is essential.

David B. Resnik, J.D., Ph.D. wrote in What is Ethics in Research & Why is it Important?

The following is a rough and general summary of some ethical principals that various codes address*:
Honesty
Strive for honesty in all scientific communications. Honestly report data, results, methods and procedures, and publication status. Do not fabricate, falsify, or misrepresent data. Do not deceive colleagues, research sponsors, or the public.
Objectivity
Strive to avoid bias in experimental design, data analysis, data interpretation, peer review, personnel decisions, grant writing, expert testimony, and other aspects of research where objectivity is expected or required. Avoid or minimize bias or self-deception. Disclose personal or financial interests that may affect research.
Integrity
Keep your promises and agreements; act with sincerity; strive for consistency of thought and action.
Carefulness
Avoid careless errors and negligence; carefully and critically examine your own work and the work of your peers. Keep good records of research activities, such as data collection, research design, and correspondence with agencies or journals.
Openness
Share data, results, ideas, tools, resources. Be open to criticism and new ideas.
Respect for Intellectual Property
Honor patents, copyrights, and other forms of intellectual property. Do not use unpublished data, methods, or results without permission. Give proper acknowledgement or credit for all contributions to research. Never plagiarize.
Confidentiality
Protect confidential communications, such as papers or grants submitted for publication, personnel records, trade or military secrets, and patient records.
Responsible Publication
Publish in order to advance research and scholarship, not to advance just your own career. Avoid wasteful and duplicative publication.
Responsible Mentoring
Help to educate, mentor, and advise students. Promote their welfare and allow them to make their own decisions.
Respect for colleagues
Respect your colleagues and treat them fairly.
Social Responsibility
Strive to promote social good and prevent or mitigate social harms through research, public education, and advocacy.
Non-Discrimination
Avoid discrimination against colleagues or students on the basis of sex, race, ethnicity, or other factors not related to scientific competence and integrity.
Competence
Maintain and improve your own professional competence and expertise through lifelong education and learning; take steps to promote competence in science as a whole.
Legality
Know and obey relevant laws and institutional and governmental policies.
Animal Care
Show proper respect and care for animals when using them in research. Do not conduct unnecessary or poorly designed animal experiments.
Human Subjects Protection
When conducting research on human subjects, minimize harms and risks and maximize benefits; respect human dignity, privacy, and autonomy; take special precautions with vulnerable populations; and strive to distribute the benefits and burdens of research fairly.
* Adapted from Shamoo A and Resnik D. 2015. Responsible Conduct of Research, 3rd ed. (New York: Oxford University Press). https://www.niehs.nih.gov/research/resources/bioethics/whatis/index.cfm

Research often forms the basis of policy decisions and laws. It is vital that research used as the basis of those decisions be conducted as far as possible in a manner free of bias and in accord with ethical research guidelines.

Where information leads to Hope. © Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART©
http://drwildaoldfart.wordpress.com/

Dr. Wilda Reviews ©
http://drwildareviews.wordpress.com/

Dr. Wilda ©
https://drwilda.com/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: