Standards Australia Updates Biometrics Performance Testing.
Standards Australia Updates Biometrics – Standards Australia recently updated part one of its biometric performance testing and reporting standard (AS ISO/IEC 19795.1:2022, Information technology – Biometric performance testing and reporting, Part 1: Principles and framework).
The revision provides guidance on achieving unbiased and accurate reporting, while introducing a level playing field for vendor testing and evaluation.
Standards Australia Updates Biometrics-The performance of biometric systems can be affected by various factors, including the composition of the training data, the matching algorithm and the interpretation of the matching errors. Simply put, data can be skewed towards certain characteristics if the associated biometric system is trained on a less diverse data set.
Facial recognition software has been criticised for the potential to be biased, and testing has found this to be the case. To evaluate the performance of a biometric system the US National Institute of Standards and Technology (NIST) conducts large-scale Facial Recognition Vendor Tests (FRVT) on multiple algorithms under various test scenarios. The tests provide a benchmark to vendors and potential users of the performance of the algorithms.
There are numerous factors that can cause errors in biometric systems, in both capturing and enrolling the biometric data. For example, errors can occur when the samples collected are either poor or damaged, which may be a result of the capture device or the environment.
According to Stacy Oerder, chair of Standards Australia IT-032 committee, Biometrics and Identification, the risk of these errors mean that the biometric system won’t work as intended for the user, or the user won’t be able to use the system at all – it’s here that AS ISO/IEC 19795.1:2022 can help.
“The importance of this standard cannot be overstated,” Oerder said. “It provides a set of principles and guidelines on testing to introduce a level playing field for all vendor testing, which enables the comparison of products.
“Without standards and practices like this, potential customers or implementers are dependent on their suppliers providing unbiased reporting on performance and capabilities.”
AS ISO/ IEC 197951:2022 specifies requirements on test protocols with the intended goal to reduce bias due to inappropriate data collection or analytic procedures.
The standard creates the general principle for testing the performance of biometric systems with reference to the error rates and throughput rates with specified performance requirements.
It aims to identify performance metrics for biometrics systems, help achieve the best estimate of field performance for the expended effort and enhance understanding of the limits of applicability of the test results.
“The updated standard informs people of the performance algorithms for both verification and identification and promotes uniform testing of algorithms,” Oerder said.
“By using biometrics, we had to know how algorithms match as well as how biometric systems performed. The complexity that we have seen over time is because algorithms are getting smarter and better.”
You can find AS ISO/IEC 19795.1:2022 via the Standards Store and distribution partners here or check out more SEN news here.
“Standards Australia Updates Biometrics Performance Testing.”
