The Iris Exchange (IREX) III report is the first public and independent comparison of commercially available algorithms that use iris recognition for the challenging task of finding an individual match within a large database of potential identities. Previous published studies only used single algorithms or considered one-to-one verification, in which an individual claims an identity and the software then attempts to confirm whether the claim matches a specific record.
NIST evaluated 92 different iris recognition algorithms from nine private companies and two university labs, all of which submitted software to an open competition held by NIST. The task was to identify individuals from a database of eye images taken from more than 2.2 million people.
“If, for example, you are trying to pick out a fugitive who is trying to cross a national border, you need to know your software can identify that person from among millions of records,” says Patrick Grother, a scientist in NIST’s Information Access Division.
“This ability to pick out a ‘needle in a haystack’ quickly and accurately is crucial, and we found some algorithms can search a haystack thousands of times larger than others. This is important because often there is no corresponding record, no needle to be found.”
Among the results accuracy varied across the algorithms the NIST team tested. Success rates ranged between 90 and 99 percent among the algorithms, meaning that no software was perfect, and some produced as many as 10x more errors than others.
Also, the tests found that while some algorithms would be fast enough to run through a dataset equivalent to the size of the entire U.S. population in less than 10 seconds using a typical computer, there could be limitations to their accuracy.
A related NIST report showed that accuracy could be improved if operators control image collection more tightly during acquisition, thereby obtaining better quality iris images.
“When combined with the feedback that this study provides to the industry and the use of the iris in combination with other biometrics, the findings will push accuracy toward 100 per cent,” Grother says.
