Skip to content
 

Brought to you by: 

Government's Use of Facial Recognition Under Scrutiny

IRS decides to allow other ID verification methods after privacy advocates raise alarms

a man holds a smart phone and it performs a scan of his face

Getty Images

En español

Facial recognition has long been a divisive issue.

Proponents say the technology is an effective tool for catching bad guys and verifying who we say we are. Critics counter that it is an invasion of privacy, potentially inaccurate and racially biased — and may even result in wrongful arrests. Because of the complexity of the issue, AARP has not taken a position at this time.

As a consumer, you may welcome the ease of unlocking your smartphone by showing your face. However, you may be less keen about the prospect of having to upload a selfie and use your mug to access tax data online. Yet that was what the Internal Revenue Service had been asking people to do — until the agency announced a change in plans in early February.

IRS moves away from facial recognition

Under a controversial new identity verification process that had been scheduled to kick in this summer, taxpayers who wanted to log in to IRS.gov would have provided a photo from a driver’s license, state ID or passport and have it matched against an uploaded selfie captured via smartphone or webcam.

The IRS was planning to use a private Virginia-based verification company called ID.me that has its facial recognition technology in use at other federal agencies and in 30 states, in some instances to verify the identities of people filing unemployment insurance claims.

ID.me says its technology helps combat fraud perpetrated by people using fake identities, and that it has prevented the loss of hundreds of billions of dollars in government benefits over the past 18 months. The company’s accounts are also used to verify the identity of military veterans so that they can earn discounts on products ranging from Oakley sunglasses to HP laptops.

“Unlike a Social Security number or credit card number, you can never change your face.”​​

— Jennifer Lynch, Electronic Frontier Foundation

But critics such as Caitlin Seeley George, campaign director at the Boston-based digital advocacy group Fight for the Future, called for the IRS to halt the plan.

“This is an example of just how facial recognition has spread widely to a point where it is touching so many parts of really everyone’s lives,” George said before the IRS announcement. “In this specific case, it’s really raised to a level of concern because everyone has to do taxes.”

She worried that a database of biometric scans, images of government IDs and those selfies would “more than likely be shared with different departments within the government” and also become a huge target for hackers, she said. George was concerned that ID.me’s privacy language was “obviously very vague and unsettling,” suggesting that it retains the right to share data with other partners under certain circumstances.


AARP Membership -Join AARP for just $9 per year when you sign up for a 5-year term

Join today and save 43% off the standard annual rate. Get instant access to discounts, programs, services, and the information you need to benefit every area of your life. 


ID.me declined an AARP request for an interview. Instead, the company directed AARP to its blog, where a distinction was made between one-to-one (1:1) facial recognition, which is similar to the way you unlock a smartphone, and one-to-many (1:many), in which a photo of your face is compared with others in a database.

“Selfies are critical to verifying identity and preventing fraud,” the company wrote. “We use ‘1:1’ matching for identity verification, and ‘1:many’ for fraud detection. We do not share selfies with any government agency unless fraud is detected.”

ID.me added that “1:many is internal to ID.me and does not involve any external or government database. It occurs once during enrollment and exists to make sure a single attacker is not registering multiple identities. The selfie is turned into a mathematical representation of a face and then compared against multiple accounts to see if a single person has registered multiple different identities that do not belong to them.”

As part of a LinkedIn post in late January, ID.me founder and CEO Blake Hall wrote: “We avoid disclosing methods we use to stop identity theft and organized crime as it jeopardizes their effectiveness.”

But after the IRS backed away from its facial recognition plans, Hall also retreated. He announced that ID.me would provide its government clients an alternative verification option that does not involve facial recognition.

New manual review offered

ID.me in late February put together what it is calling a Human in the Loop team of reviewers to weigh in if an ID card or selfie match is rejected during the automated process with the company’s government partners. If the information passes this manual review, users can continue identity verification through the automated process. If the document or selfie doesn't pass muster, then users are directed to a one-to-one live video chat to verify with a human agent.

“For years, we have combined human agents with automated technology to guard against any potential bias,” Hall said in a statement. “Now, the Human in the Loop team enables us to verify more users faster in order to ensure equitable and streamlined access to vital government services.”

ID.me says early results show the Human in the Loop approach is reducing the number of users who need to verify through video chat by 18 percent.

Hall has also said that ID.me users will be able to delete their selfies or photos at account.ID.me starting March 1.

Databases can be linked

Surveillance litigation director Jennifer Lynch of the nonprofit Electronic Frontier Foundation (EFF), which defends civil liberties in the digital world, says she has been equally troubled by the IRS’ plans and by facial recognition more broadly.

"Unlike a Social Security number or credit card number, you can never change your face,” she says. “Once we are all in a face recognition database, then it becomes possible to attach those databases to camera networks that are out in society and identify people as they walk about through society.”

Think about the possibility of, for example, discovering the name and other details of an attractive stranger you pass on the street.

IRS looks to balance privacy, fraud protection

Fight for the Future, the Algorithmic Justice League, the Electronic Privacy Information Center (EPIC) and other civil rights organizations launched a website calling for people to take action against the IRS plan. In announcing plans to develop an additional online verification process that doesn't involve facial recognition, IRS Commissioner Chuck Rettig acknowledged the concerns that were raised.

“The IRS is consistently looking for ways to make the filing process more secure, but to be clear, no American is required to take a selfie in order to file their tax return,” the U.S. Treasury Department said in a statement. “We believe in the importance of protecting the privacy of taxpayers while also ensuring criminals are not able to gain access [to] taxpayer accounts.

“The lack of funding for IRS IT modernization has made it impossible for the IRS to invest in state-of-the-art technology. The IRS today uses third-party service providers to validate the identification of individuals attempting to improperly gain access to taxpayer accounts,” the Treasury Department said. “This includes ID.me, which is compliant with the National Institute of Security Technology standards and used by multiple agencies across the government.”

​“We believe in the importance of protecting the privacy of taxpayers while also ensuring criminals are not able to gain access [to] taxpayer accounts.”

— U.S. Treasury Department statement​

The public outcry over facial recognition certainly predates the IRS situation. In 2020, Pope Francis raised ethical concerns around the technology. Several U.S. states and cities have enacted bans or limited its use by the police. Tech giants Amazon, Facebook, IBM and Microsoft have all backed away from facial recognition, at least to some degree.

This past November, Facebook announced it would delete more than a billion individual facial recognition templates, meaning those on the social network who opted into a Face Recognition setting will no longer be recognized automatically in photos and videos. Still, Facebook is reluctant to give up on the tech forever.

“Looking ahead, we still see facial recognition technology as a powerful tool, for example, for people needing to verify their identity, or to prevent fraud and impersonation,” Jérôme Pesenti, who heads artificial intelligence efforts at Facebook parent Meta, wrote in a blog entry. “We believe facial recognition can help for products like these with privacy, transparency and control in place, so you decide if and how your face is used. We will continue working on these technologies and engaging outside experts.”

Fighting romance scams, finding missing persons

One company still deeply committed to facial recognition is Clearview AI, whose software is used by law enforcement. Cofounder and Chief Executive Hoan Ton-That cited cases where Clearview’s facial recognition has helped catch criminals preying on the elderly through romance scams. He also brought up a case where the technology helped authorities identify a disoriented woman with dementia who was wandering around a convenience store without an ID.

“The process would have been a lot longer had they used fingerprinting that might not have worked, or waited for a missing persons report,” Ton-That told AARP.

Clearview’s software searches the entire internet for publicly available photos, thus Ton-That’s description of a kind of “Google for faces,” a database of more than 20 billion images. Pictures may be picked up from social media, school websites, news sites and so on.

Clearview’s “hit rate” is 75 percent, meaning 75 percent of the time law enforcement is getting a true match, says Ton-That. For the other 25 percent, “we are showing no results for search. We’d rather show nothing than show a false positive.”

Bias in facial recognition

Various studies through the years have pointed to biases inherent in some facial recognition algorithms, resulting in error rates that have been higher for women of color, older adults and children in some cases. Ton-That, who is half Asian and half Australian, says he’s mindful of bias.

“We trained [our algorithm] on so many examples of faces from every single ethnicity from the open internet,” he says. Clearview's accuracy rates across various demographics are well above 99 percent on tests that the U.S. government’s National Institute of Standards and Technology conducted.

Ton-That adds that the photos are not admissible in court and merely provide a lead to law enforcement in “after-the-fact investigations,” not in real time. Clearview does not sell a version of its software to the public.

Clearview has been in the crosshairs of privacy advocates for some time. It does not sell a version of its software to the public.

Banned from selling to most businesses

In May, Clearview will be permanently banned from selling its face database to most American businesses and “other private entities” after settling a lawsuit filed by the American Civil Liberties Union in Illinois. Clearview also agreed to end the practice of offering free trial accounts to individual police officers without the approval of their bosses.

“Clearview can no longer treat people’s unique biometric identifiers as an unrestricted source of profit. Other companies would be wise to take note, and other states should follow Illinois’ lead in enacting strong biometric privacy laws,” Nathan Freed Wessler, a deputy director of the ACLU Speech, Privacy, and Technology Project, said in a statement. 

Clearview’s database is still available to government agencies.

“We put certain controls in place around the technology to make sure law enforcement uses it appropriately and [officers] don’t use it as a sole source of an arrest or investigation,” Ton-That told AARP months before the settlement.

ID systems are hard to use

Lynch of EFF brings up an issue that may concern some older people. The systems are “designed by people who have access to and sophistication with the latest technologies on our cell phones and computers.

“But they are then used by people who are using older technologies, who may have disabilities, and who may have challenges working with the modern technologies of today,” she says. “Then you get into what we’re seeing with IRS and some of the state agencies. You get into this loop where you actually can’t verify your own identity.”

Even people who are technically proficient encounter obstacles. Security expert Brian Krebs purposely went through the drill of creating an ID.me account.

“Successfully verifying your identity with ID.me may require a significant investment of time and quite a bit of patience,” he wrote on his KrebsOnSecurity blog. “For example, stepping away from one part of the many-step application process for a little more than 5 minutes necessitated another login, and then the re-submission of documents I’d previously uploaded.”

Technology outpaces existing law

Facial recognition as an option for verification of a person’s identity may come down to lawmakers’ actions.

“We actually need to have a public debate about an overarching comprehensive federal law dealing with biometric data privacy before we even consider moving forward [with] particularly large-scale uses like the IRS [was] intending to do in their relationship with ID.me,” says Jeramie D. Scott, senior counsel at EPIC. “Although there are positive uses for facial recognition technology, it is a very invasive and powerful surveillance technology that is really easy, and therefore very tempting, to use by the government as well as companies.”

This story, originally published Feb. 3, was updated to reflect the ACLU’s Illinois lawsuit and a change in IRS plans and ID.me’s decision to provide an alternative verification option and a manual review process.

Edward C. Baig is a contributing writer who covers technology and other consumer topics. He previously worked for USA Today, BusinessWeek, U.S. News & World Report and Fortune, and is the author of Macs for Dummies and the coauthor of iPhone for Dummies and iPad for Dummies.​​

More on Privacy and Personal Technology