Testing done late this spring by the Department of Homeland Security’s research and development division on the use of biometric technology to identify travelers in high-throughput checkpoint environments demonstrated the importance of finding the right pairing of biometric capture system and identity matching algorithm for optimum performance, according to DHS Science and Technology (S&T).
Of the 97 tests of various system combinations, which included multiple face, iris and fingerprint capture systems and multiple face, iris and fingerprint matching algorithms, just 12 met the goals of the test, an industry official supporting S&T said on webinar hosted by the agency in August for the Biometric Technology Rally participants. The webinar was archived and released publicly on Sept. 4.
“And that should give some concern because all systems included in this evaluation passed subject matter expert review for inclusion into the rally and met all the rally participation criteria,” Yevgeniy Sirotin, who works for Science Applications International Corp. [SAIC] in support of S&T, said during the webinar. “So, if you’re putting together an operational system deployment and you’re looking for acquisition systems and algorithm matching systems to kind of combine, without some extra testing, your chances of success might be closer to 12 percent, which is not very encouraging.”
In the case of the system combinations that met the rally goal, all were face matching and face acquisitions systems, he said. The goal was 99 percent true identification of the volunteer test subjects.
The biometric rally, conducted in May at an S&T facility in Maryland near Washington, D.C., was the second conducted by S&T in as many years. The recent test included 14 biometric acquisition systems and 15 matching systems selected by an interagency review panel for inclusion in the event. In the published results, S&T uses aliases to describe the performance of each vendors’ systems.
One vendor’s camera system used for acquiring facial images, which was given the alias Teton, achieved 100 percent true identification with four matching algorithms and a 96.5 percent rate with a fifth algorithm. No other vendor’s camera system achieved a 100 percent identification rate with any other vendor’s face matching algorithm.
Four face matching algorithms met the rally goal of at least 99 percent true identification in combination with at least one acquisition system.
S&T said that 430 volunteers participated in the six-day rally, with each subject going through all facets of the test regime. The station for each test was unmanned so the volunteers had to interact with the technology based on system prompts.
“When you conduct screening operations, you usually have an officer involved. If you have a ratio of one officer to one person, that’s very time consuming and expensive,” Arun Vemury, director of S&T’s Biometric and Identity Technology Center, said in a statement. “These technologies, the way they were tested, no one was providing instruction to the user, so operators can focus on higher value security tasks. It was supposed to be intuitive and easy to someone who wants to participate but may not be familiar with the new process.”
After a face, iris or fingerprint image was captured, a volunteer exited the station and went to a kiosk to provide a satisfaction score for the collection systems using one of four buttons ranging from very happy to very unhappy.
The rally goal for satisfaction scores was greater than 95 percent and the minimum satisfaction threshold was greater than 90 percent. Only five acquisition systems, all face capture cameras, met the goal. Two additional face capture systems met the minimum threshold as did one fingerprint capture system and one multimodal system that captured face and iris images.
The rally also tallied efficiency and effectiveness metrics for the participating vendors’ biometric capture systems. Of the 14 acquisition systems evaluated, seven achieved the efficiency goal, which was less than five seconds on average for test volunteers to perform a transaction with each system. Of these seven, six were face capture system and one a fingerprint capture device.
“We found that several use cases, such as border security, aviation security, and physical access controls at government facilities have similarities. The intent here is to continue to push technology developers to continue to improve technologies, make them easier, faster, and more accurate,” Vemury said. “We are working with technology providers to clarify our expectations that the technology work more effectively for all users, reduce errors, make systems more cost-effective for governments, and offer a better user experience for the people undergoing the screening process.”
Within DHS, Customs and Border Protection uses fingerprint checks for all arriving foreign nationals to the U.S. to verify their identities. The agency is transitioning to automated facial imaging checks for all arriving international air travelers and is rolling out facial imaging checks for all departing international air travelers. The Transportation Security Administration is also evaluating automated facial image checks for travelers at airport security checkpoints.