How Facial Recognition Works: The Ghost in the Camera
Facial recognition research by online companies like Facebook, commercial vendors, and universities is making the technology faster and more accurate
Facial recognition is not a single technology. Instead, it’s a broad field in which researchers use 3D modeling, analysis of patterns of light and dark in photographs, and other techniques to first pick out faces from a video stream or still photo, then identify either characteristics of the subject (male or female, age range, race) or a specific identity.
The most widely used technique relies on taking hundreds of measurements between established facial features. One leading vendor of the technology is Cognitec Systems, which in the past several years has expanded from its home offices in Dresden, Germany, into the U.S., Australia, and other countries. Elke Oberg is the company’s marketing manager.
“Essentially what is being looked at is a landscape of the face,” Oberg says. “Facial recognition software takes various measurements of each face and turns these into a string of numbers. Then it’s just a matter of comparing one string of numbers with another. The higher the similarity score, the more likely it is that you’re looking at the same person.” The resulting file is called a faceprint or face template—it can consist of thousands of digits, depending on what algorithm is used. Faceprints can be compared with databases for a wide range of purposes: to recognize shoplifters, verify identities to open electronically controlled gates, or simply count how many people are standing in a particular line or crowding around a popular store display.
A facial recognition research project called DeepFace that was conducted by Facebook and described in a paper in summer 2014 used a computing architecture called a deep neural network. The project was an example of “machine learning.” Researchers didn’t tell the computer to take a predetermined set of measurements of each photo. Instead, they built a system that automatically analyzed millions of images, turned them into 3D models, and then figured out on its own how to pick out which photographs matched.
The system was 97.35 percent accurate when applied to a publicly available dataset of more than 13,000 photographs collected from online news stories with uneven lighting, shot from a variety of angles.
That kind of work has the potential to make facial recognition systems faster, more accurate, and easier to scale up to handle huge numbers of images. The computer would know which individuals appeared in almost any photo, taken almost anywhere—and do it almost instantly.
Editor's Note: This article also appeared in the February 2016 issue of Consumer Reports magazine.
