How DNA evidence went from airtight to error-prone

Faye Flam
Bloomberg

Blind faith in any technology can be dangerous — especially when it comes to areas of forensic science such as DNA fingerprinting. For example, if police have “DNA evidence” against a suspect, most juries will assume that’s proof of guilt. But while the technology for analyzing DNA has become vastly more sensitive since it was first introduced in courts in the 1990s, crime labs are working with ever more minute traces — sometimes just a few molecules —and drawing inconsistent or erroneous conclusions from them. In fact, there’s good reason to believe DNA evidence has sent people to prison for crimes they didn’t commit.
That’s the conclusion of a recent report commissioned by the President’s Council of Advisors on Science and Technology (PCAST), which called into question the increasingly common practice of analyzing mixtures of DNA from several individuals. Police can now collect DNA not only from blood stains and other bodily fluids, but from traces of cells left behind when people touch door handles, guns or other objects associated with crime scenes. These so-called “touch DNA” samples often include a mixture of genetic material from multiple people who handled those items.
The report didn’t just focus on DNA analysis — it surveyed a variety of forensic techniques, such as bite-mark and footprint analysis, and it found many scientifically lacking. Labs were inconsistent in their results and grossly underestimated how likely they were to be wrong. But the problems with DNA analysis are particularly worrisome, since jurors have such strong faith in its accuracy.
Analyzing mixtures of DNA is “not like a Ouija board — it’s not junk science in that sense,” said David Kaye, a law professor at Penn State University and author several books on forensic science. Scientists can extract useful information from those mixtures, he said. But he agrees with the report that right now there’s no reliable, consistent protocol for interpreting that information. “Forensic science is not regulated the way clinical medical labs are,” Kaye said. “And people are being put to death.”
DNA is reliable when it’s analyzed correctly, there’s a large enough sample, and it comes from just one person. The traditional system uses 13 different sites on the chromosomes where the genetic code tends to hiccup — five, 10 or maybe 20 times. The number of hiccups in each area varies from person to person. If two samples match the number of hiccups at all 13 sites, the odds of them coming different people are one in billions. That changes when DNA in a sample is a mixture. Drawing inferences from these is much shakier. The report cites several shocking cases, including one in which a prosecutor told a jury the chance of a mistaken DNA match was one in 1.1 billion. A later analysis of the evidence put it at “closer to one in two.”
Molecular geneticist Greg Hampikian believes poor mixture analysis led to a wrongful conviction of a Georgia man named Kerry Robinson. A woman who had been raped by several men identified one suspect, who was offered a deal if he could identify his accomplices. That man made several guesses, said Hampikian, and eventually named Robinson. While the victim couldn’t identify him, Robinson was convicted and sent to prison based on DNA evidence.
Hampikian, who volunteers as head the Idaho Innocence Project, examined the DNA, and concluded that it not only failed to implicate Robinson, it excluded him. He decided to use the case to prove a wider point, taking the same DNA samples to another lab and asking 17 different experts to analyze it. Only one concluded that the DNA might have come from Robinson. All the others said it was inconclusive or excluded him.
Hampikian’s study was cited in the PCAST report. Robinson remains in prison. Another problem Hampikian finds with DNA is that the use of minuscule samples, often just a few cells, makes contamination hard to avoid. Such a trace analysis led to the famous conviction of American exchange student Amanda Knox, who was accused of murdering a woman she had roomed with in Italy. Knox and her Italian boyfriend were both sent to prison based on a fragment of DNA allegedly belonging to the victim found on a knife pulled from the boyfriend’s kitchen drawer.
Independent Italian scientists reviewed the case on appeal and concluded that the DNA was most likely transferred through contamination. Knox’s conviction was overturned — but not until she had already served four years in prison. Illustrating the problem, Hampikian had his students at Boise State University pick up soda cans left in an office after the lunch hour and put them in evidence bags. Later, they performed DNA analysis on newly purchased knives. The DNA from a faculty member appeared on the knives, even though nobody but the students had been near them.
All science is inherently uncertain, but good scientists know how to measure and express that uncertainty. In contrast, Hampikian said he often hears expert witnesses use phrases such as “high degree of scientific certainty,” or “match.” His view is echoed in the report, which says that it’s “not scientifically defensible” to claim any conclusion “100 percent certain” or to say an error rate is “zero,” “essentially zero” or “negligible.”
While some journalists have labeled the techniques in the report “junk science,” the report never used that phrase. Nor did it recommend scrapping any practices. Instead, the panel recommended independent testing to determine how well the techniques work.
Unfortunately, scientists seem to be more enthusiastic about those conclusions than law enforcement. The FBI objected that the report makes “broad, unsupported assertions,” and Attorney General Loretta Lynch countered “the current legal standards regarding the admissibility of forensic evidence are based on sound science.”
But the report is a valuable gift from eminent scientists to the FBI. Both, after all, are professional seekers of truth. Scientists know that some degree of error in inevitable, and that overconfidence can lead them astray. That’s why they have methods designed to overcome bias, acknowledge uncertainty, and correct their mistakes. In this sense, the scientific method offers a lesson for law enforcement: Admitting you could be wrong is often the first step toward getting things right.