Tag Archives: Law

National Institute of Standards and Technology study: Courtroom use of ‘Likelihood Ratio’ not consistently supported by scientific reasoning approach

15 Oct

Virginia Hughes wrote in How Many People Are Wrongly Convicted? Researchers Do the Math.:

So then the next terrifying question is, geeze, how many innocent people have actually been executed?
Fortunately it’s probably not many. Innocent defendants are far more likley to have their sentenced changed to life in prison than to be executed. Still, with an error rate of 4 percent, the researchers write, “it is all but certain that several of the 1,320 defendants executed since 1977 were innocent.”
It’s impossible to say whether this 4.1 percent false conviction rate applies to defendants who never went to death row. But I’ll leave you with one last depressing thought. Of all of the people found guilty of capital murder, less than half actually get a death-penalty sentence. And when juries are determining whether to send a defendant to death row or to life in prison, surveys show that they tend to choose life sentences when they have “residual doubt” about the defendant’s guilt.
That means, then, that the rate of innocent defendants serving life in prison is higher than those on death row. “They are sentenced,” the authors write, “and then forgotten….” http://phenomena.nationalgeographic.com/2014/04/28/how-many-people-are-wrongly-convicted-researchers-do-the-math/

See, More Innocent People on Death Row Than Estimated: Study http://time.com/79572/more-innocent-people-on-death-row-than-estimated-study/

Anders Nordgaard and Birgitta Rasmusson wrote in Law, Probability and Risk, Volume 11, Issue 4, 1 December 2012:

Abstract
The ability of the experienced forensic scientist to evaluate his or her results given the circumstances and propositions in a particular case and present this to the court in a clear and concise way is very important for the legal process. Court officials can neither be expected to be able to interpret scientific data, nor is it their task to do so (in our opinion). The duty of the court is rather to perform the ultimate evidence evaluation of all the information in the case combined, including police reports, statements from suspects and victims, witness reports forensic expert statements, etc. Without the aid of the forensic expert, valuable forensic results may be overlooked or misinterpreted in this process. The scientific framework for forensic interpretation stems from Bayesian theory. The resulting likelihood ratio, which may be expressed using a verbal or a numerical scale, compares how frequent are the obtained results given that one of the propositions holds with how frequent they are given that the other proposition holds. A common misunderstanding is that this approach must be restricted to forensic areas such as DNA evidence where extensive background information is present in the form of comprehensive databases. In this article we argue that the approach with likelihood ratios is equally applicable in areas where the results rely on scientific background data combined with the knowledge and experience of the forensic scientist. In such forensic areas the scale of the likelihood ratio may be rougher compared to a DNA case, but the information that is conveyed by the likelihood ratio may nevertheless be highly valuable for the court. https://academic.oup.com/lpr/article-abstract/11/4/303/931682/The-likelihood-ratio-as-value-of-evidence-more

See, On the interpretation of likelihood ratios in forensic science evidence: Presentation formats and the weak evidence effect. https://www.ncbi.nlm.nih.gov/pubmed/24814330

Science Daily reported in Courtroom use of ‘Likelihood Ratio’ not consistently supported by scientific reasoning approach:

Two experts at the National Institute of Standards and Technology (NIST) are calling into question a method of presenting evidence in courtrooms, arguing that it risks allowing personal preference to creep into expert testimony and potentially distorts evidence for a jury.
The method involves the use of Likelihood Ratio (LR), a statistical tool that gives experts a shorthand way to communicate their assessment of how strongly forensic evidence, such as a fingerprint or DNA sample, can be tied to a suspect. In essence, LR allows a forensics expert to boil down a potentially complicated set of circumstances into a number — providing a pathway for experts to concisely express their conclusions based on a logical and coherent framework. LR’s proponents say it is appropriate for courtroom use; some even argue that it is the only appropriate method by which an expert should explain evidence to jurors or attorneys.
However, in a new paper published in the Journal of Research of the National Institute of Standards and Technology, statisticians Steve Lund and Hari Iyer caution that the justification for using LR in courtrooms is flawed. The justification is founded on a reasoning approach called Bayesian decision theory, which has long been used by the scientific community to create logic-based statements of probability. But Lund and Iyer argue that while Bayesian reasoning works well in personal decision making, it breaks down in situations where information must be conveyed from one person to another such as in courtroom testimony.
These findings could contribute to the discussion among forensic scientists regarding LR, which is increasingly used in criminal courts in the U.S. and Europe.
While the NIST authors stop short of stating that LR ought not to be employed whatsoever, they caution that using it as a one-size-fits-all method for describing the weight of evidence risks conclusions being driven more by unsubstantiated assumptions than by actual data. They recommend using LR only in cases where a probability-based model is warranted. Last year’s report from the President’s Council of Advisors on Science and Technology (PCAST) mentions some of these situations, such as the evaluation
Bayesian reasoning is a structured way of evaluating and re-evaluating a situation as new evidence comes up. If a child who rarely eats sweets says he did not eat the last piece of blueberry pie, his older sister might initially think it unlikely that he did, but if she spies a bit of blue stain on his shirt, she might adjust that likelihood upward. Applying a rigorous version of this approach to complex forensic evidence allows an expert to come up with a logic-based numerical LR that makes sense to the expert as an individual.
The trouble arises when other people — such as jurors — are instructed to incorporate the expert’s LR into their own decision-making. An expert’s judgment often involves complicated statistical techniques that can give different LRs depending on which expert is making the judgment. As a result, one expert’s specific LR number can differ substantially from another’s.
“Two people can employ Bayesian reasoning correctly and come up with two substantially different answers,” Lund said. “Which answer should you believe, if you’re a juror?”
Viewpoints differ on the appropriateness of using LR in court. Some of these differences stem from the view that jurors primarily need a tool to help them to determine reasonable doubt, not particular degrees of certainty. To Christophe Champod, a professor of forensic science at the University of Lausanne, Switzerland, an argument over LR’s statistical purity overlooks what is most important to a jury….
The NIST authors, however, maintain that for a technique to be broadly applicable, it needs to be based on measurements that can be replicated. In this regard, LR often falls short, according to the authors.
“Our success in forensic science depends on our ability to measure well. The anticipated use of LR in the courtroom treats it like it’s a universally observable quantity, no matter who measures it,” Lund said. “But it’s not a standardized measurement. By its own definition, there is no true LR that can be shared, and the differences between any two individual LRs may be substantial.”
The NIST authors do not state that LR is always problematic; it may be suitable in situations where LR assessments from any two people would differ inconsequentially. Their paper offers a framework for making such assessments, including examples for applying them.
Ultimately, the authors contend it is important for experts to be open to other, more suitable science-based approaches rather than using LR indiscriminately. Because these other methods are still under development, the danger is that the criminal justice system could treat the matter as settled…. https://www.sciencedaily.com/news/health_medicine/

Citation:

Caution in use of courtroom evidence presentation methods urged
Courtroom use of ‘Likelihood Ratio’ not consistently supported by scientific reasoning approach
Date:
October 12, 2017
Source:
National Institute of Standards and Technology (NIST)
Summary:
Two experts are calling into question a shorthand method of presenting forensic evidence in courtrooms, arguing that it risks allowing personal preference to creep into expert testimony and potentially distorts evidence for a jury.
Journal Reference:
1. Steven P. Lund, Hari Iyer. Likelihood Ratio as Weight of Forensic Evidence: A Closer Look. Journal of Research of the National Institute of Standards and Technology, 2017; 122 DOI: 10.6028/jres.122.027

Here is the press release from NIST:

NIST Experts Urge Caution in Use of Courtroom Evidence Presentation Method
Use of ‘Likelihood Ratio’ not consistently supported by scientific reasoning approach, authors state.
October 12, 2017
Two experts at the National Institute of Standards and Technology (NIST) are calling into question a method of presenting evidence in courtrooms, arguing that it risks allowing personal preference to creep into expert testimony and potentially distorts evidence for a jury.
The method involves the use of Likelihood Ratio (LR), a statistical tool that gives experts a shorthand way to communicate their assessment of how strongly forensic evidence, such as a fingerprint or DNA sample, can be tied to a suspect. In essence, LR allows a forensics expert to boil down a potentially complicated set of circumstances into a number—providing a pathway for experts to concisely express their conclusions based on a logical and coherent framework. LR’s proponents say it is appropriate for courtroom use; some even argue that it is the only appropriate method by which an expert should explain evidence to jurors or attorneys.
However, in a new paper published in the Journal of Research of the National Institute of Standards and Technology (link is external), statisticians Steve Lund and Hari Iyer caution that the justification for using LR in courtrooms is flawed. The justification is founded on a reasoning approach called Bayesian decision theory (link is external), which has long been used by the scientific community to create logic-based statements of probability. But Lund and Iyer argue that while Bayesian reasoning works well in personal decision making, it breaks down in situations where information must be conveyed from one person to another such as in courtroom testimony.
These findings could contribute to the discussion among forensic scientists regarding LR, which is increasingly used in criminal courts in the U.S. and Europe.
While the NIST authors stop short of stating that LR ought not to be employed whatsoever, they caution that using it as a one-size-fits-all method for describing the weight of evidence risks conclusions being driven more by unsubstantiated assumptions than by actual data. They recommend using LR only in cases where a probability-based model is warranted. Last year’s report (link is external) from the President’s Council of Advisors on Science and Technology (PCAST) mentions some of these situations, such as the evaluation of high-quality samples of DNA from a single source.
“We are not suggesting that LR should never be used in court, but its envisioned role as the default or exclusive way to transfer information is unjustified,” Lund said. “Bayesian theory does not support using an expert’s opinion, even when expressed numerically, as a universal weight of evidence. Among different ways of presenting information, it has not been shown that LR is most appropriate.”
Bayesian reasoning is a structured way of evaluating and re-evaluating a situation as new evidence comes up. If a child who rarely eats sweets says he did not eat the last piece of blueberry pie, his older sister might initially think it unlikely that he did, but if she spies a bit of blue stain on his shirt, she might adjust that likelihood upward. Applying a rigorous version of this approach to complex forensic evidence allows an expert to come up with a logic-based numerical LR that makes sense to the expert as an individual.
The trouble arises when other people—such as jurors—are instructed to incorporate the expert’s LR into their own decision-making. An expert’s judgment often involves complicated statistical techniques that can give different LRs depending on which expert is making the judgment. As a result, one expert’s specific LR number can differ substantially from another’s.
“Two people can employ Bayesian reasoning correctly and come up with two substantially different answers,” Lund said. “Which answer should you believe, if you’re a juror?”
In the blueberry pie example, imagine a jury had to rely on expert testimony to determine the probability that the stain came from a specific pie. Two different experts could be completely consistent with Bayesian theory, but one could testify to, say, an LR of 50 and another to an LR of 500—the difference stemming from their own statistical approaches and knowledge bases. But if jurors were to hear 50 rather than 500, it could lead them to make a different ultimate decision.
Viewpoints differ on the appropriateness of using LR in court. Some of these differences stem from the view that jurors primarily need a tool to help them to determine reasonable doubt, not particular degrees of certainty. To Christophe Champod, a professor of forensic science at the University of Lausanne, Switzerland, an argument over LR’s statistical purity overlooks what is most important to a jury.
“We’re a bit presumptuous as expert witnesses that our testimony matters that much,” Champod said. “LR could perhaps be more statistically pure in the grand scheme, but it’s not the most significant factor. Transparency is. What matters is telling the jury what the basis of our testimony is, where our data comes from, and why we judge it the way we do.”
The NIST authors, however, maintain that for a technique to be broadly applicable, it needs to be based on measurements that can be replicated. In this regard, LR often falls short, according to the authors.
“Our success in forensic science depends on our ability to measure well. The anticipated use of LR in the courtroom treats it like it’s a universally observable quantity, no matter who measures it,” Lund said. “But it’s not a standardized measurement. By its own definition, there is no true LR that can be shared, and the differences between any two individual LRs may be substantial.”
The NIST authors do not state that LR is always problematic; it may be suitable in situations where LR assessments from any two people would differ inconsequentially. Their paper offers a framework for making such assessments, including examples for applying them.
Ultimately, the authors contend it is important for experts to be open to other, more suitable science-based approaches rather than using LR indiscriminately. Because these other methods are still under development, the danger is that the criminal justice system could treat the matter as settled.
“Just because we have a tool, we should not assume it’s good enough,” Lund said. “We should continue looking for the most effective way to communicate the weight of evidence to a nonexpert audience.”
Paper: S.P. Lund and H. Iyer, Likelihood Ratio as Weight of Forensic Evidence: A Closer Look. Journal of Research of National Institute of Standards and Technology, Published online 12 October 2017. DOI: 10.6028/jres.122.027 (link is external)
Forensic Science, Mathematics & Statistics, Public safety and Standards
Media Contact
Chad Boutin
boutin@nist.gov (link sends e-mail)
(301) 975-4261
Related News
• How to Quantify the Weight of Forensic Evidence: A Lively Debate
The ability to afford to challenge evidence often determines the outcome of very serious charges.

Mathew Shaer wrote in the The False Promise of DNA Testing: The forensic technique is becoming ever more common—and ever less reliable.:

Modern forensic science is in the midst of a great reckoning. Since a series of high-profile legal challenges in the 1990s increased scrutiny of forensic evidence, a range of long-standing crime-lab methods have been deflated or outright debunked. Bite-mark analysis—a kind of dental fingerprinting that dates back to the Salem witch trials—is now widely considered unreliable; the “uniqueness and reproducibility” of ballistics testing has been called into question by the National Research Council. In 2004, the FBI was forced to issue an apology after it incorrectly connected an Oregon attorney named Brandon Mayfield to that spring’s train bombings in Madrid, on the basis of a “100 percent” match to partial fingerprints found on plastic bags containing detonator devices. Last year, the bureau admitted that it had reviewed testimony by its microscopic-hair-comparison analysts and found errors in at least 90 percent of the cases. A thorough investigation is now under way…. https://www.theatlantic.com/magazine/archive/2016/06/a-reasonable-doubt/480747/

The reliability of the evidence and the ability of a particular accused to defend against evidence presented in a court hearing is crucial to preventing the innocent from being convicted.

Where information leads to Hope. © Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART©
http://drwildaoldfart.wordpress.com/

Dr. Wilda Reviews ©
http://drwildareviews.wordpress.com/

Dr. Wilda ©
https://drwilda.com/

Advertisements

The 11/16/13 Joy Jar

16 Nov

Moi had the great pleasure and honor of volunteering as a judge at the University of Washington Law School’s Moot Court competition today. There were many other volunteers. Many organizations rely on volunteers to carry out their message and mission. Today’s deposit into the ‘Joy Jar’ is the millions of volunteers who make the world a more civilized and better place.

The heart of a volunteer is not measured in size, but by the depth of the commitment to make a difference in the lives of others.
DeAnn Hollis

Volunteers do not necessarily have the time; they just have the heart.
Elizabeth Andrew

Volunteering is not a choice, it’s a responsibility.
Unknown

Never doubt that a small group of commited people can change the world. Indeed, it is the only thing that ever has.
Margaret Mead

If you want to lift yourself up, lift up someone else.
Booker T. Washington

A civilization flourishes when people plant trees under which they will never sit.
Greek Proverb

The best way to find yourself, is to lose yourself in the service of others.
Ghandi

You make a living by what you get. You make a life by what you give.
Winston Churchill

Don’t ever question the value of volunteers. Noah’s Ark was built by volunteers; the Titanic was built by professionals.
Unknown

Volunteerism is the voice of the people put into action. These actions shape and mold the present into a future of which we can all be proud.
Helen Dyer

Without community service, we would not have a strong quality of life. It’s important to the person who serves as well as the recipient. It’s the way in which we ourselves grow and develop…
Dr. Dorothy I. Height

Volunteers are the only human beings on the face of the earth who reflect this nation’s compassion, unselfish caring, patience, and just plain love for one another.
Erma Bombeck

Volunteering creates a national character in which the community and the nation take on a spirit of compassion, comradeship and confidence.
Brian O’Connell

No need to kill all the lawyers, they are dying of natural causes

14 Nov

Here’s today’s COMMENT FROM AN OLD FART: Moi is an Emeritus Attorney, she practices pro bono law and no longer works for fees. She still gets the bar magazine called the NW Lawyer and this month’s edition is designed to smack the barristers into reality. First, there is the article, The First Thing We Do: Let’s Kill All The Lawyers http://nwlawyer.wsba.org/nwlawyer/november_2013#pg26

The real reality check comes from Washington State Bar Association (WSBA) President Patrick A. Palace as he opines on the state of the profession in the article, Game Changers at page six:

The Changing Profession

Some lawyers blame the loss of business on the economy and are waiting for the good times to return. Others complain that they are under attack by unauthorized practitioners and we need to build more protections, but most agree that practicing law is getting harder. And there are specific reasons practice is getting harder. http://nwlawyer.wsba.org/nwlawyer/november_2013#pg9

Palace goes on to describe the challenges of law practice in detail and offer some possible paths to surviving the environment. This is a must read for anyone considering a career as a lawyer

The career conundrum starts early. Kelsey Sheehy posted at U.S. News the article, Study: High School Grads Choosing Wrong College Majors:

Teens are selecting college majors that don’t match their interests, according to a new report, but experts say that is OK.

Choosing the right college major is almost as important as choosing the right college.

Majors give students direction and allow them to map out their path to graduation. Some majors, such as engineering, information technology or accounting, also help prepare students to enter a specific career field.

Students who select a major that matches their interests are more likely to stick with it and finish their degree on time, but few high school graduates are choosing a major that suits them, a report released today by ACT Inc. finds.

Almost 80 percent of ACT test-takers who graduated in 2013 said they knew which major they would pursue in college. Of those students, only 36 percent chose a major that fit their interests, according to the study. ACT used answers from the exam’s Interest Inventory, which asks a series of questions to determine career areas where a student might excel.

This disconnect isn’t exactly surprising, says Beth Heaton, senior director of educational consulting at College Coach, an advising firm.

A former regional director of admissions for the University of Pennsylvania, Heaton has read thousands of college applications. She now advises teens trying to get into college.

“The vast majority of them have no idea what they really want to do when they grow up. Even the ones who claim that they do,” she says. “How can you know? If you’re 16, 17, 18, you know so little of the world….” http://www.usnews.com/education/blogs/high-school-notes/2013/11/11/study-high-school-grads-choosing-wrong-college-majors

Here is the press release from ACT:

Many Students Select a College Major That Doesn’t Fit Their Interests Well

November 11 2013

IOWA CITY, Iowa—Only about a third of ACT®-tested high school graduates selected a college major that is a good fit with their interests, according to a new research report from ACT titled College Choice Report Part 1: Preferences and Prospects. The findings suggest that high school students need more help planning for college and career—and that the majority want such assistance.

ACT’s research looked at the 79 percent of ACT-tested 2013 high school graduates who reported a college major that they planned to pursue, comparing their ACT Interest Inventory profile with profiles of college students in the same program of study. Students complete the ACT Interest Inventory when registering for the exam. Interest profiles for students with the same majors were based on a national sample of undergraduate students with a declared major and a grade point average of at least 2.0.

Although the majority of ACT-tested graduates selected a major that was at least a moderate fit with their interests, only 36 percent selected one that was a good fit, while nearly as many—32 percent—selected a major that was a poor fit with their interests.

“It’s important for students to have the information they need to make the best decisions about their future,” said Jon Erickson, ACT president of education and career solutions. “They should be made aware that choosing a college major that reflects their interests will give them a better chance of succeeding and could also contribute to their satisfaction and happiness in school and on the job.”

Previous ACT research has suggested that students whose interests are similar to those of people in their chosen college majors are more likely to remain in their major, persist in college and complete their degree in a timely manner. Students who change their major while in college may have to take additional courses to satisfy degree requirements or even transfer to a different institution, potentially delaying their graduation.

ACT’s report indicates that a student’s intended major can play a significant role in which college he or she chooses to attend. Half of those students who indicated a planned major reported that the availability of a particular major was the most important factor in selecting a college.

“Students who start out with the right major choice can save significant time and money, which is increasingly important given the rising cost of attending college,” said Erickson. “Far too many colleges require students to select a major without looking at how well the students’ interests fit with their intended program of study.”

The ACT report suggests that students would welcome help with their planning. Around three out of five ACT-tested 2013 graduates indicated that they needed assistance with their educational and occupational plans. Among students who were undecided on a college major, the number indicating that they needed assistance rose to more than seven out of ten.

“We must do more to help students connect their majors and, ultimately, their careers to their interests, so they can be on a path for success,” said Erickson. “ACT offers a number of free resources to help students plan for college and career, and we are working to develop more to help in this area.”

ACT’s free resources include the World-of-Work Map, which is included with every ACT score report, and the new ACT Profile, the first college and career readiness social community. ACT’s website for students, http://www.actstudent.org, offers other free college and career planning resources, including tips, checklists, and information on topics such as academic preparation, choosing a college major, choosing a college, and applying to colleges.

ACT will release the College Choice Report Part 2 in July 2014. http://www.act.org/newsroom/releases/view.php?lang=english&p=3064#.UoUmc3rYZWA.

Moi is not picking on the legal profession, but she is making a point that what might have looked like an optimum career path in past years may not be the path for the future. Dennis Smith has a good brief article at College Recruiter. Com, Choosing A Vocation: Finding Your Calling

“What do you want to do with your life?”

I’ve heard everything from,

“I want to be the VP of Engineering!”, to “I don’t really know what I want to do….I only know what I don’t want to do.”

In my opinion, both answers are good. I’ve known engineers that knew they were going to be engineers from their mother’s womb. I’ve known others who, like myself, enjoy doing so many different things that they graduate from college not having made specific plans for the day after graduation.

In making this decision, the mistake made by many of us is that we too often listen to the multitude of voices that are willing to offer up advice about what “we” should be doing with our lives. As my grandfather used to say, “That advice and a nickel will get you a cup of coffee.

What is it that matters most? What is it that you want to do more than anything? What makes you truly happy? What is it that makes you “alive?

Curt Rosengren says,

“If there’s one thing I’ve discovered over the years, it’s that just about anything we set our minds to is possible. Moreover, one of the biggest – if not the biggest – obstacle we face lies smack dab between our ears. We’re so often overcome with fear of what might go wrong that we don’t dare to even take a step.” “But….what would you do if you were brave?”

Students should be thinking about what is the appropriate life balance for them. http://www.collegerecruiter.com/blog/2006/12/12/choosing-a-vocation-finding-your-calling/

Another important part of career or vocational selection is life balance.

WebMD and the Mayo Clinic have some good suggestions about life balance.

WebMD advises in 5 Tips for Better Work-Life Balance:

1. Figure Out What Really Matters to You in Life

2. Drop Unnecessary Activities

3. Protect Your Private Time

4. Accept Help to Balance Your Life

5. Plan Fun and Relaxation

http://www.webmd.com/health-insurance/protect-health-13/balance-life

The Mayo Clinic has tips for striking the proper work-life balance http://www.mayoclinic.com/health/work-life-balance/WL00056/NSECTIONGROUP=2

A man’s work is nothing but this slow trek to rediscover, through the detours of art, those two or three great and simple images in whose presence his heart first opened.

Albert Camus

After two weeks of working on a project, you know whether it will work or not.

Bill Budge

All labor that uplifts humanity has dignity and importance and should be undertaken with painstaking excellence.

Martin Luther King, Jr.

Where information leads to Hope. © Dr. Wilda.com

Dr. Wilda says this about that ©

Blogs by Dr. Wilda:

COMMENTS FROM AN OLD FART© http://drwildaoldfart.wordpress.com/

Dr. Wilda Reviews © http://drwildareviews.wordpress.com/

Dr. Wilda © https://drwilda.com/