The myth of fingerprints

I've been reading Ross Anderson's epic tome, Security Engineering: A Guide to Building Dependable Distributed Systems (2nd edition, 2008, Wiley), and have just gotten into the chapter on biometrics (ch. 15).  Section 15.5.2, on Crime Scene Forensics, points out three major criminal cases where fingerprint matches have been in error, including the Brandon Mayfield case which I wrote about at this blog back in 2007.  Anderson points out that law enforcement agencies have claimed to juries "that forensic results are error-free when FBI proficiency exams have long had an error rate of about one percent, and misleading contextual information can push this up to ten percent or more" (pp. 470-471).  It's probability at work:

Even if the probability of a false match on sixteen points [the UK standard, the U.S. has no minimum] were one in ten billion (10^-10) as claimed by police optimists, once many prints are compared against each other, probability theory starts to bite. A system that worked fine in the old days as a crime scene print would be compared manually with the records of a hundred and fifty-seven known local burglars, breaks down once thousands of prints are compared every year with an online database of millions. (p. 471)

One of the other two cases Anderson discusses is that of Scottish policewoman Shirley McKie, who was prosecuted on the basis of a 16-point fingerprint match found at a murder scene and could not find any fingerprint examiner in Britain to defend her.  She found two Americans who testified on her behalf that it was not a match (Anderson shows the crime scene print and her inked print on p. 469; the crime scene print is heavily smudged).  McKie's own fellow officers tried to convince her to give false testimony about her presence at the crime scene, which she refused to do.  She was acquitted, but lost her job and was unable to get reinstated.

The third case Anderson mentions is Stephan Cowans, who was convicted of shooting a police officer after a robbery in 1997.  He was convicted, but argued it was not his fingerprint.  After Cowans was able to get crime scene evidence tested for DNA which was found not to match, a re-examination of the fingerprint also found that there was no match.  So six years after his conviction, he was acquitted on appeal.

Further evidence of the errors which can arise from fingerprint examination comes from two studies by psychologist Itiel Dror which Anderson describes.  In one study, five fingerprint examiners were each shown a pair of prints, allegedly the falsely matched prints from the Mayfield case, and asked to point out the errors.  Three examiners gave explanations for the non-matches, one said that they did, in fact, match, and one was uncertain.  In fact, the pairs of prints were each purported matches by the corresponding examiner from a recent criminal case, so only one of the five was still certain that a match testified to in court was in fact a match upon re-examination with a skeptical mindset.  In a second study, Dror gave each of six experts eight prints that they had matched in previous cases, and four of the six gave inconsistent results.

Anderson points out that belief in the infallibility of fingerprint evidence has the effect of promoting carelessness by examiners, not giving proper critical scrutiny to the method or its assumptions in changing conditions (e.g., the increase in the number of fingerprints to match against in the age of the computer), and increasing the negative consequences of cases of failure.  In the McKie case, Anderson points out, "there appears to have arisen a hierarchical risk-averse culture in which no one wanted to rock the boat, so examiners were predisposed to confirm identifications made by colleagues (especially senior colleagues).  This risk aversion backfired when four of them were tried for perjury." (p. 472)

Itiel Dror's two papers (references from Anderson, p. 923):

IE Dror, D Charlton, AE Péron, "Contextual information renders experts vulnerable to making erroneous identifications," in Forensic Science International 156 (2006) 74-78

IE Dror, D Charlton, "Why Experts Make Errors," in Journal of Forensic Identification v 56 no 4 (2006) pp 600-616; at http://users.ecs.soton.ac.uk/id/biometrics.html

(Previously, which includes reference to Simon Cole's book on fingerprint evidence which shares the title of this post.)

Fingerprint-matching pseudoscience

Fingerprint matching has been used as an investigative tool by law enforcement and as a key piece of evidence to convict criminals in courts for over a century, but its accuracy has not actually been scientifically tested until recently. It turns out that claims of its accuracy have been exaggerated, and fingerprint matching is often more art than science.

In 2004, the FBI claimed that a fingerprint found on a bag at the sign of a terrorist bombing in Madrid, Spain on March 11 was a match to the left index finger of Brandon Mayfield, an attorney in Beaverton, Oregon who converted to Islam and married to a Muslim woman from Egypt. Despite the fact that Spanish police disagreed, claiming that there was no match to Mayfield, the FBI insisted they had a "one hundred percent identification" with fifteen separate points of agreement between the latent print from Spain and Mayfield's fingerprint, validated by at least three FBI fingerprint examiners. Mayfield was arrested and detained on May 6, 2004. On May 20, Spanish police announced that they had matched the fingerprint to Ouhnane Daoud of Algeria, who--unlike Mayfield--had actually been in Spain. Mayfield was released and the FBI ended up apologizing.

This case has resulted in scientific scrutiny of fingerprint evidence that has been long overdue. A decade ago, Tucson printer and publisher of the anarchist periodical The Match!, Fred Woodworth, published "A Printer Looks at Fingerprints," in which he pointed out pseudoscientific reasoning in fingerprint matching methodology as described in fingerprint textbooks. Subsequently, Simon Cole authored the book Suspect Identities: A History of Fingerprinting and Criminal Identification (2002, Harvard University Press), and has just authored an article on the subject in the July/August 2007 issue of Skeptical Inquirer, titled "The Fingerprint Controversy."

In Cole's Skeptical Inquirer article, he states that "The very first study containing accuracy data was finally published just recently, finding very high accuracy rates in a class of trainees on latent prints of unknown difficulty; but the study contains some methodological flaws (Haber and Haver 2006). Moreover, the authors again argue strongly against inferring accuracy rates from their own data (Wertheim, Langenburg, and Moenssens 2006)."

No doubt scientific investigation will demonstrate that proper use of fingerprint analysis is a reliable method of identification, but more importantly, it will find its limits and weaknesses so that it does not continue to be pressed beyond its capabilities and result in false judgments of guilt in criminal cases. Unfortunately, law enforcement and prosecutors have a vested interest in the flexibility of techniques that can be used to produce the judgment they want, as demonstrated by the difficulty in getting police departments to modify their procedures of eyewitness identification of suspects to correct for well-known cognitive biases.