Facial recognition software has gotten 20 times better at searching databases of facial photographs to find a matching image, according to the National Institute of Standards and Technology’s (NIST) recent evaluation of 127 software algorithms from 45 different developers—the bulk of the industry. This indicates there is a rapidly advancing marketplace for face-based biometric algorithms.
The evaluation was based on the same test that was carried out in 2010 and 2014 to determine how well an algorithm can match a person’s photo with a different one of the same person stored in a large database. This type of “one to many” search is often used to check for a person who might be applying for a visa or driver’s license under a name different than their own.
The algorithms were prototypes, submitted in 2018 as compiled libraries and were evaluated as black boxes behind a NIST-specified C++ testing interface. The report therefore cannot not describe exactly how algorithms operate.
The test used four databases containing 30.2 million photographs of 14.4 million individuals. It included images of twins, candid photos (so called “non-cooperative subjects,” and images of the same person taken several years apart. This makes it the largest public and independent evaluation of face recognition ever conducted. The participants did not have access to the databases of photos, which NIST kept from developers. The test also evaluated software for how well it handled the effects of aging on faces, identifying twins, how it worked on poor-quality images, and its scalability to large databases.
The test results include data on accuracy, speed, investigative vs. identification applications, and the algorithm’s scalability to large populations.
The team found that just 0.2% of searches failed this year, compared with a 5% failure rate in 2010 and 4% in 2014. Failure means that when an image of a person’s face is submitted to the recognition software, it fails to return the matching face image in the database.
All of the top-performing algorithms from the latest round make use machine-learning software architectures called convolutional neural networks. According to NIST’s Patrick Grother, the rapid advance of machine-learning tools has effectively revolutionized the industry.
“The fact that error rates have fallen this far implies that end users will need to update their technology,” says Grother. “The test shows a wholesale uptake by the industry of convolutional neural networks, which didn’t exist five years ago. And about 25 developers have algorithms that outperform the most accurate one we reported in 2014.”
But prospective users should beware: The new algorithms do not all perform the same, and the best algorithms are far ahead of the pack.
“There remains a very wide spread of capability across the industry,” Grother notes. “This implies you need to properly consider accuracy when selecting new-generation software.”
NIST’s report identifies the submissions by name and presents them in ranked tables. The main ranked list reflects how often an algorithm put the correct result of the “one to many” search at the top of the list of possible identities, a metric called the rank one recognition rate.
However, the report also considers variations such as how the algorithms performed when misidentifications, called false positives, must be minimized. Other algorithms sometimes outperformed the best performing algorithm when these variant factors took priority.
